I can't choose just one. Once you have kids, it's fun dragging them around places and seeing things...so they'll have an understanding of the world through experiences. We took our kids (3 and 5) to the Smithsonian a few weeks back. It was actually pretty neat seeing the taxodermy animals, the art, the planes/space vehicles, etc... We'll go back in a few years, so they remember more of it. I took them to a planetarium last weekend solo for the sake of learning.
We've taken them to Disney World and different beaches on quite a few trips.... Speaking Disney, I had only been twice before and only for a few days....I never really spent a lot of time there. We went there for a conference at the Grand Floridian with a 2.5 year old and 6 month old. It was a great trip...pushing the stroller, riding all the rides in MK except the mountains. Eating tons of good food. 2 kid to 2 parent ratio is great and we manage those trips well....knowing expectations, food, and potty breaks.
Of course, a trip with just me and my wife is nice. We did a solo trip for 3-4 days before meeting up with the kids and extended family in our condo/timeshare at the beach. It was a pretty nice week and a half trip for us.
The key is to avoid a group that's too large (for the sake of dining, transportation, and extra occupancy issues). There are certain family members and friends you have to just learn to avoid. People who are too particular about things or control freaks are the worst. Sometimes the best part of vacation is planning for a lot of possibilities, but choosing the one the group wants to go for. I've known some people that want to deviate from the laid-back approach and that's counter-productive. I'd rather just roll with it.