Some people who are against gay marriage are homophobic, and some are not. Just b/c you think someone is in the wrong does not mean you hate them or think poorly of them. If that were the case, everyone would have to hate everyone else b/c every person does wrong every day. For those who base their self-worth on thinking they are better than other people, this would be more of an issue.
To be honest, I've never heard a single argument against gay marriage that wasn't either of the form "it's weird so I don't like it" or was rooted in religion. While they may not be homophobic, they are definitely bigots.