That isn't necessarily it--at least not all of it.
First and foremost is Brady can be viewed as a beneficiary of the "Patriot Way." Player after player was churned over as a commodity (Bill mastered the new NFL before everyone else imo) with the exception of Brady. Huge stars were pretty much cast off when their cost/benefit analysis wasn't deemed in the Patriot's favor.
Which goes back to his first 4 seasons when they won 3 Super Bowls. Brady wasn't deemed a top tier quarterback like Farve, Manning, Warner, Gannon, McNabb, or even a McNair or Garcia. Brady was viewed -- MUCH LIKE RUSSELL WILSON -- to be the beneficiary of a great team.
(IMO it is overlooked that great QB play isn't limited to attempts/yards/touchdowns but also turnovers and taking risks when needed to get the key first down, not necessarily volume. Brady was good about playing within his system/limits unlike other top tier QBs who either by their nature couldn't control themselves [Farve, who I love] or had to make up for their teams).
Part of the hate is the tuck rule. He didn't make the rule but he benefited. Which goes back to the "he is more lucky than good."
Winning a couple Super Bowls by thin margins against teams like Carolina doesn't help. Nor the fact a kicker (!) winning an MVP over the QB.
The perception of Bill cheating before the Ram game hurts, too. Not just the background chatter of taping but the actual game. Everyone wants to talk about how Seattle holds and PIs (and hence the rule changes) even if a number of studies show they didn't do so more than others. But Bill absolutely took advantage of the soft playoff officiating and had his DBs mugging the Colts and Rams. It was obnoxious and not normal (despite what Patriot fans will tell you) for the time. It isn't a foul if the refs don't call it. This was a "let them play" ref mentality and it didn't sit well with a lot of non-Boston area fans. Brady benefited from such.
Brady also isn't always viewed (right or wrong) as a very nice "guy" in regards to football. Part of this is a projection of the Patriot way. Part of that is his berating of teammates. Comments about hold outs. Saying f*** on TV like he did during the Green Bay game.
And it didn't help his image our America when the headlines were he cheated/dumped his pregnant girl friend for a Super Model. Doesn't even matter what the truth it. Even without a lot of women fans (at the time) guys wives/girlfriends saying, "Hey, do you know that famous QB guy who always wins?" He got a girl pregnant and cheated on her! What a scum bag." The obvious result is no sane man would root for Brady with that gal around. You don't counter back, "Well, in Tom's defense, she was a crazy woman and they weren't really together." WRONG ANSWER.
There is some of that. But that is far from the entire Tom Brady story.