This is a difficult problem and in my mind boils down to some market-based fundamentals.
As long as the government does not preferentially distribute permits or fine/tax business, then I can see a strong argument for allowing private discrimination. If I imagine a small business such as a convenience store or restaurant, then I would simply say that discrimination would be foolish, but not dangerous. If this was prevalent in a few communities, I can even imagine myself going to the area to cater to the newly displaced customers. It seems like a fairly obvious proposition.
Then I think about markets without such mobility. The larger the corporation gets, the more imperfect the market, the bigger the impact of these decisions. What if the town only has one mechanic or one private clinic or one super store (Target, Wal-Mart, etc.)? The barriers to entry rise and the odds of a competitor decrease. What if private industries in one sector coordinate to deny service to a particular minority? Is that allowed (assuming they still compete for their allowed class of customer)?
In other words, in the presence of an efficient market, I don't think anti-discrimination laws are required. The more inefficient the market becomes, the more difficult it becomes to oppose anti-discrimination laws.
I'm not sure where that line should be drawn or if it can be.