Your advocacy is showing pretty strongly here, keys.
You're right that 32mm2 by itself isn't going to make or break Nvidia. It is going to hurt them though. It's not just the slightly bigger die, it's also coupled with the somewhat lower yields. Based on the fact that GF104 has only shown itself in a moderately harvested version, the default assumption has to be that yields aren't that great right now. I love how you just assume GF104 is going to be a higher volume part than Cypress with nothing to back it up. Fact is, cypress is a smaller die, covers a larger range of the market, has significant market penetration according to the steam hardware surveys, and is already very mature with very good yields, and ATI now has more marketshare than Nvidia as well. Nvidia has a lot of work to do if they want to catch up to cypress-level volume.
The significant thing about the GF104 die size is that it means Nvidia is pricing a more expensive part (and yes, it is obviously more expensive right now vs. the smaller more mature more mass produced cypress core. In the future that may or may not change, but right now it is undeniably more expensive) significantly below what AMD is pricing their stuff at. This isn't a part that is going to make huge amounts of money, it's positioned as a marketshare bandage part. The only issue is that soon enough GF104 is going to be the only competitive part Nvidia has left and it isn't going to be a cash cow. Southern Islands is probably going to obsolete GF100, and CPU-integrated graphics in SB and Fusion is going to instantly evaporate Nvidia's low end. That leaves them stuck in the middle with GF104, and at that time they'll need to sell the fully enabled part at $200 because of pressure from SI and derivatives. They still have the pro market, but if the rumors of AMD coming out with much better performing drives for firepro are true, that might start eroding as well.
So, the point is that Nvidia is going to be locked out of the high and low ends fairly quickly, and their mid-range card, while priced and positioned well for consumers, probably isn't going to be enough to stop Nvidia from losing a lot of money.