Whoa, only few hours and so many replies, thanks people. Now, the error is not in me, just look at the picture on the page. the top of the vertical line. I started from bottom, and as you can see it changed the direction very straight. A hand could not draw that kind of line just like that. It's too straight to be 'my hand'.
Oh and about trying on other machines, I have two computers, see? So far I haven't noticed any problems with the mice on that one...
Now...I remember trying with a ball mouse (like loosbrew suggested) and both of the axises 'go wacko' so it's not the mouse (that should be pretty obvious, thinking that I tried 3 different mice)
I have also tried different surfaces so that can't be it either. Tried table surface, mouse pad, paper etc.
If you check the message board I mentioned on the page I have added a few new things I've tried. I welcome all new ideas as I'm running out of options here.
I'm gonna try moving this thing to another room later, in case there's something in this room that could be interfering, though I doubt it. And the other machine I have isn't affected, so...