There are a few separate things going on here, so in no specific order:
1) The X1K cards offered a variable level of acceleration given the task at hand. For MPEG2 it was pretty close to complete offloading, only a few bits were missing. The X1K series in particular offered some of the
best acceleration out there for the time. For H.264 however none of the cards that could do only partial acceleration were particularly good, you'd only save
10-25% on your CPU utilization. The point for this being is that you're not going to get tremendously low CPU usage with your X1800XT doing the acceleration.
2) To use the acceleration features, a decoder must know how to use DXVA. Platform specific decoders like PowerDVD's decoder module and Media Player Classic HC's decoder module can do this. Cross platform decoders like VLC (or rather libavcodec) can not. Furthermore libavcodec is particularly poor because it's not multithreaded for H.264. Realistically it can only use a single core plus a few percent from the others.
3) You're just low on CPU power period. VLC's not going to cut it, you can try a DXVA compliant decoder but I don't know how much farther you're going to get. Your best bet may be to try out CoreAVC, a software-only decoder with particularly low CPU usage (in exchange for not-quite-to-spec decoding). You may need to disable deblocking/post-processing to squeeze out enough CPU power.