well... the speakers sound the same when plugged into the jacks, but the headphones clearly don't. Now that I think about it, it sounds more like a jump from stereo to mono when I switch the headphones than a loss in quality. The sound just sounds more flat, if that makes sense.
to answer your question, i'm not listening to them both simultaneously 😛 I prefer the speakers for most stuff but occasionally I like the phones and sometimes it's necessary not to disturb the rest of the family (i.e., when playing HL2 at 1am)
edit: forgot to ask, are all splitters created equal or do some have better quality?