All valid points. I will add, in reference to your hypothetical about dumping toxics into a river, that there is no remedy for this among consumers because of the basic collective action problem. If company A dumps toxic waste because it is cheap and expedient to do so, and company B - a competitor of company A - is ethical and incurs the expense of proper waste disposal, then company A's product is cheaper.Obviously. Most natural monopolies, for example. Joe Blow in the sticks would never see electricity, much less internet, because the cost to set up the wiring would not be profitable to the electric company until way way way down the road if ever, compared to immediate profits from other uses for that investment capital. Same goes for roads to every home, postal service to every home. Not everyone will be able to afford schools, and those catering to the poor by definition will have to aim for a high volume, lower quality model.
Then there's the fundamental problems of externalities and information flow. There's no profit-motivated way to deal with company A dumping into the river and making people sick 20 miles downstream. Even if the people strongly suspected that was the cause, the needed investment in time and money to exhaustively research the water chemical levels along the river for 20 miles or more would put it well out of the reach of any given private citizen, or even voluntary collective of citizens. And if they DO prove it, so what? Company A can just sell to the people upstream or in the next territory who haven't heard of the pollution or (more likely) don't care because it doesn't affect them.
Similarly, when company B starts using questionable 'meat' in their sausages that don't make people immediately sick, but build up and cause illness down the road, how are consumers to know (and therefore use the only recourse they have in this scenario, choosing to not buy that sausage)? They'd have to have home FDAs.
Information is expensive in time and often money, certainly in money if we start privatizing all sources of it, and reliable information costs even more time to verify. Not everyone has the time to invest into information gathering to make fully-informed rational choices, even assuming the mythical rational decision-maker existed. We need regulations to give us a baseline of knowledge on which we can act - that our foods are almost always up to a certain standard, same with our water, and any special dangers will be listed for us to decide on.
It is in fact illogical for a consumer to buy company B's product over company A's cheaper product out of protest of company A's behavior, because each individual consumer knows that his purchase decision is irrelevant to the manufacturer's bottom line. He has no guaranty that other consumers will follow suit, and even if they do, then his decision to not boycott company A will make no difference either. The logical choice is to buy the cheaper product, assuming the products are otherwise identical. And this is the logical choice for every other consumer as well. The only way it is rational to avoid the product is where consuming the product itself causes harm, yet as you say, this requires first that the consumer have knowledge of the harm.
The sole remedy, that of after-the-fact litigation initiated by those directly harmed, is inadequate for a slew of reasons. You already mentioned that the expense may be prohibitive. Yet even if it isn't prohibitive, the harm has already been done. Injuries can be compensated for by monetary damages, but who ever said that this is an adequate remedy, particularly for deaths and other serious injuries?
And it isn't going to be a deterrent to future bad behavior either, assuming the cost of litigation and/or settling each claim is less than the added expense of proper waste disposal. This is often the calculus in these situations. If company A saved $1 billion last year by dumping into the river, then so what if they had to settle with 50 families for $2 million apiece? The logical choice for the company is to give their kids leukemia then settle up, not to incur the expense of proper disposal.
And notice how in all of this, company B is penalized for being ethical? The incentive in an unregulated market is to do the expedient and cost-efficient thing, regardless of the harm it causes.
There is no other alternative except to regulate business to prevent these kinds of harms from occurring to begin with.