It's a word in informal useage, but not allowed in formal useage. However, it's still a word.
Think of having to dress up for work. Even though you can't wear your tye-dye shirt to work, it's still a shirt. It's just not a shirt for work.
To say a contraction is not a word is a selective bias based on individual ideosyncracies and environmental mores. It's still a signal for communication, a verbal composition, therefore it has to be word--unless it wasn't something in common useage. These things are a transitory thing. A while ago "bling" wasn't a word because of its limited social dispersion, but it's understood now and is in common useage. It is now a word.