No. We're inundated with sporting events and for the most part see the athletes we care about on a consistent basis. I am not a fan of swimming/gynmastics/track & field, but the reasons I watch are for the sob/special stories.
As far as I know the U.S. has been at the top if not #1 in medals for quite a while. Always winning somewhat undermines nationalistic pride as we just expect to always win, so we only perk our noses up when we lose. This is in contrast with some country who wins 5 medals, that's huge for them and their country...they are heroes.
Michael Phelps was the last olympics (and probably this one), but how many americans would recognize his face?
Even those extraordinary stories are forgotten and replaced in 4 yrs.
---
I think that americans would need the olympics if the country lost its pride...something akin to a horrible war, gas rationing or an economic depression. For us, it is important when we need something uplifting in our lives. If you don't need any uplifting, then you probably don't care at all.