In addition to being heavily biased toward DirectX9 cards, it's also very dependent on your driver set. My rig scored 2335 with the ATI Catalyst 2.5 installed, but 3445 with Catalyst 3.0! The problem is that it won't run the Nature test with the 2.5 drivers, thus drastically lowering your score (i.e. just like all the NVidia cards). Contrast that to the difference between 11350 and 11500 scores in 3DMark 2001 SE for those two drivers, respectively. I haven't tried it with the Catalyst 3.1 yet, since my refresh forcing tweak won't work with the latest driver set and I can't stand 60Hz (I run Windows ME, and it won't recognize my monitor properly without the tweak).
Certainly, it doesn't seem to be a very relevant benchmark for 99% of the cards out there today. But I suppose 3DMark 2001 didn't match actual game performance when it first came out either. In the long run, that benchmark became more relevant with time .... but I have my doubts about 3DMark03 given its very limited repetoire of testing.
My rig:
Asus A7V8X
AMD XP2000+ (stock)
ATI Radeon 9500 Pro (stock)
512mb PC2700 DDR RAM
WD 40GB
SB Audigy