OP, ditch the football and just start up a NO MA*AM chapter instead. No excuse for any guy not to go then.
As for my opinion, I'm not a big sports fan, I can watch it very very casually if I have too, but I much rather do something else with my free time then pretend to give a crap about watching some rich guys playing a game.
I doubt I'd go to the party either. Since the party is center around football (regardless if it was meant too or not, from a guest perspective it is), I know I don't want to be around a bunch of sports fans where that is going to be the topic of focus mostly.
My best friend holds a super bowl party each year, he knows how I feel at sports and gives the invite out of courtesy but knows I probably wont show up. I went a few times simply out of courtesy as well, but if I have anything even remotely more appealing to do I'll that instead.
As for the person who says they can't understand people who no interest in sports. I kinda wonder the same thing about hardcore sports fans that have every piece of junk with their teams logo/name on it and how they get some emotionally invested in a game, especially one over about of people you don't actually know and don't give a crap about you.