Obviously there's tradeoffs of using headphones versus speakers and vice versa.
But even speakers can't properly replicate various feeling, like the concussion from an explosion or a helicopter landing nearby. It can give you an impression of such, but unless you use ridiculously powerful speakers its not going to match it. That's all speakers really are though, is attempting to give you an adequate impression of what the recording is. I would also say that some recordings really play to the inherent strengths of headphones, and others to those of speakers.
We're a long way off as far as real tactile feedback is concerned. For me personally, its just as easy to let my mind simulate those effects in the intimacy and isolation that headphones offer as far as sound. I end up with more overall immersion.
As always, it comes down to what you prefer, but I would strongly recommend making any assumptions about what you prefer before you've really had adequate experience either way. I know some people that were convinced that headphones were inferior until they heard a decent pair that cost less than their speaker setup and felt the headphones offered a much better audio experience. Likewise, just because you've only heard crappy speakers or headphones, don't write them off, as generally quality is a very significant factor in the audio experience, and many people haven't really experienced that to much degree.
If space and finances allowed for it, I would absolutely setup a designated speaker room.