There really is no "best" for me personally. I've gone on vacations just my wife and myself, then my wife and myself and a group of friends, and then just my wife and my son. Then I've gone on trips with must my boys. They all are great in their own ways.
When going by ourself, we don't have to worry about anyone elses schedule. My wife is pretty laid back and I pretty much run things on vacation as far as what we do and stuff. She likes to snorkel and hang at the beach with some drinks, but then if we want to go out on an excursion as well like rent ATV's or what not, she is cool with that too.
With friends, it's fun to hang with more people at the beach and just basically have a party atmosphere for a week with good friends. Going out to party at night is more fun when you can go with a group of people, although my wife and I still go out when it's just us 2.
With our son, it's a different kind of fun. I love that he loves being in the ocean and hanging out at the beach. Our vacation days when we've gone with him are basically hanging out at a beach in the morning, coming back and he naps, then we go back out and do something else, then come home and have a nice dinner at a restaurant, then come home and relax. I also love seeing my son experiencing new places and things, and seeing how excited he is about all of it. Typically the vacations with my son are a lot more relaxing than when it's without him.
When I've gone with just my boys, it's typically because we all want to do a bunch of shit that we wouldn't do with our wives around, which includes going fishing for a day, snorkeling, and scuba diving, etc. I've also gone on golfing trips where it's basically 3 days of golfing/drinking during the day then strip clubs at night. It's been a while since I've done those, but I prefer the ones that involve water activities way more.