That's not really accurate. A town of 5000 won't support a WalMart supercenter, so most of the customers would necessarily come from the surrounding towns, creating issues around traffic & infrastructure.
I think you are quite wrong on that. Walmart grew out of little towns. My hometown has a population of less than 9000 when we got a walmart and it is now being upgraded to a super walmart(population about 12k now).