With Vsync enabled, the graphics card waits for the monitor to refresh before starting a new frame. The key word there is 'wait'. If the graphics card waits, it's not doing anything, meaning you get less average frames per second. With Vysnc disabled, the graphics card draws as fast as possible. So, really, you are using your card optimally.
If this is all it did, many more people would choose Vsync off because it would give you performance benefits, but there are visual side effects that affect the choice, on or off, other than frame rate differences.
With Vsync enabled, you may experience slower mouse response time - if you do, it may be negligible anyway, so not too much of a problem. You also may notice jumps in frame rates. Say your refresh rate is 60Hz; you may notice jumps from 60fps down to 30fps. A jump like this is very distracting. With Vsync disabled, you may notice tearing, which is when part of the screen shows part of one frame and the other shows part of another frame. On an LCD, this is very noticeable, and I have to enable Vsync. On a CRT though, it is much less noticeable and I find it acceptable to disable Vsync.
Overall, on a CRT, I like Vsync off because my fps is more stable allowing a smoother gaming experience and tearing is minimal*, however Vsync on is more pleasing to my eyes. I do recommend Vsync on if you have a CRT that is able to display 120Hz at your chosen resolution. On an LCD however, I just can?t stand the amount of tearing and I have to enable Vsync and possibly lower settings to keep smooth gameplay.
*Tearing is minimal on a CRT once you are used to it, but going from Vsync on to Vsync off will allow you to see a lot of tearing that might frighten you away from Vsync off.