This'll get a laugh since it's a fairly obscure place most people have never heard of - Fairhope, Alabama.
It's on the eastern shore of Mobile Bay across from the city of Mobile. You're about 15-20 minutes from the beach, but not on the beach itself with all the tourist junk that comes with that. You're still on the water though with Mobile Bay. You're 15 minutes from a fairly large city in Mobile (Pensacola's also only about 30 minutes away.) The heart of town itself is gorgeous with lots of trees, walking paths and old Southern style homes (kind of like Mayberry on a bay.) It's got all the charm of a small town with all the convience of your typical outlying suburb. The problem is that by the time I retire it won't be the same. A lot of people are already discovering it and it's quickly becoming overrun.
My runner up is Durango, Colorado.