Actually, this can be caused by a number of different things. I myself was experiencing this in certain circumstances. I found two things... well three that ultimately helped me resolve this issue. For some reason occasionally even though the ram was only showing that it was at 617MB used, it would pause and page to the drive. I STILL don't know exactly why. These solutions were found over at Guru3D, and may or may not be useful. Try them at your own risk.
First, you want ot make sure that you have Fastwrite support turned off in your system bios for the AGP. You also want to disable fastwrites through the board's bios settings using something like RIvatuner. For older games you may need to turn this back on.(Diablo2 etc. I have noticed this being a need) However for almost if not all contemporary games seem to not use it, and infact suffer problems from it. The other thing to do is again use Rivatuner. Open it up, go down to the driver settings, under customize select the DirectX icon. Under Blitting turn on the first two objects.(Enable wait after blit, and Always wait after blit). These two will probably lower your 3dmark2003 score about 10-50 points, but you should see a gain in performance. The thinal thing to do is to use a program like LtcyCfg.exe(if need be I can go find a link to download it) and adjust the PCI latency of your videocard down. By default the cards are set at 248 now days because the higher priority grants them more use of the PCI bus and higher benchmark scores. But... ultimately in some things, it lowers performance by not giving up bus controll to other things. Try lowering it down to 128 or even 64.(I find 64 gives me the best performance)
What seems in contradiction about the latency config is that this seems to be almost like underclocking the card's potential. Not ot mention, I thought the AGP slot had its own DMA bus that it wasn't required to share with anything else. However, the results speak for themselves with me. Hopefully this helps you out.