In my experiance, it has to do alot with the controls of the game. My eyes/brain are accustomed to certain level of "cause & effect" from the mouse / keyboard input. This is probably from many millions of hours playing Quake (where I would be disturbed if my mouse sensitivity and or accelleration were off by just a little bit).
Now, any game in which the controls don't have the right "feel" (to my brain / eyes) causes me discomfort.
Some specific items which I'm sensitive too are
1) FOV of 90... I'm used to playing with 120 and can't stand games which don't let me tweak this
2) excessive headbob
3) any kind of "sliding" or clumsiness in the controls. Specifically, there is this game called powerslide which makes me want to puke in 30 seconds because my brain isn't expecting the side-to-side sliding of the perspective.
Well, that's my $0.02