Costco is usually at the same price point for fresh foods, but arguably, they're of a bit better quality. So if that matters to you, yes, they are cheaper.
You can't generalize with groceries, though. For instance, Whole Foods beats the prices that my nearby Harris Teeter charges for many of the same items that I buy. Last time, I was surprised to notice that the greek yogurt that I get (brand is Fage, just checked) is like 10-20 cents cheaper at WF than HT. Similarly, HT is cheaper for regular butter, but if I want their "european" variety (higher fat content, cultured), its cheaper if I go to WF and buy their store brand, and the WF store brand tastes better.
Still cant beat Safeway for canned goods, though. Beans, etc. And the Safeway Select brand is some of the best packaged bacon I've had, as well as chocolate, and coffee. Whatever they're using to source their Italian Roast, it's good enough to put in my espresso machine, and I normally spend $20+ a kilo on the Italian brands from a specialty store. Of course, at $8 per 10oz, it's not cheaper, but surprisingly as good or better than Miscela D'Oro and Lavazza.