I left for college from the same house I was brought home from the hospital to after I was born. I have not lived there since 1986, and my parents sold the house 5 years ago, but that city will always be my hometown.
The most common definitions I have seen says something like:
"Hometown is primarily used as a generic term for the city or town in which someone was born in, grew up in, or the place of someone's principal residence"
If you are born somewhere, but never lived there, I cannot see calling that your hometown. Also, I think you should only use your "principal residence" as your hometown if you did not have a particular place where your "grew up" or spent your "formidable years".
MotionMan