Loading levels will NOT be improved by adding more memory. A faster drive is the only thing that will improve load times (or going RAID 0 which I don't recommend if your system is mission critical).
You want to check the Device Manager properties for the Primary Channel of your IDE Controller. Make sure you're running at the highest DMA mode your controller and HDD is capable of (usually UDMA 5 or 6 on a modern system). Sometimes Windows will throttle you back to a lower DMA mode if you get a lot of errors on the IDE bus (which can be caused by something as mundane as a dirty lens on your CD-Rom).
SWAT 4 is a bad example to go by - it has very long load times even on high-end rigs like mine. We had a LAN party the other day and no one had any faster load times in SWAT 4 than I did (damn fun game to play COOP multiplayer BTW).
What more memory will do is improve framerates IF the game makes use of it. Most modern games (UT2K4 engine games, Doom3, HL2 engine games, BF1942) will definately improve in performance. BF1942 in particular is real memory whore - difference between night and day running 512 vs 1024Mb.
Chances are (assuming you have a decent video card) that your frame rates are better than they were with less memory, which will result in smoother game play. You may also be able to increase your resolution in some games higher than you were using before.
To find out if you're REALLY using the memory run some of your most intensive games and apps for a few hours and then open Task Manager, and go to the Performance tab.
If the PEAK Commit Charge value is over 512000 then you're making use of the extra memory.
You NEVER want the PEAK Commit Charge value to exceed the TOTAL Physical Memory value otherwise your system will be relying on the swap file and will take a major performance hit.
For serious gaming, I consider 1Gb to be the minimum right now. Not much reason to install more than that unless you use a very memory intensive app like Photoshop of VMWare.