I'm going to try the same thing soon, probably around Christmas.
It should be faster because the latency for flash is on the scale of a thousandth of a HDD's, even though the theoretical transfer speed is lower (480Mbps etc). If you have a 133x SD card, that comes out to 159.6Mbps (if the wikipedia page hasn't been vandalised recently). If you have an 80x SD card, the maximum speed should be 96Mbps.
Superfast HDDs like a Raptor will get something like 110Mbps of real burst throughput. The speed issue doesn't matter to me, because even if my flash card has lower transfer speeds, it has got to be better than my laptop's HDD.
The only real issue is how long the flash card will last before it dies. But you can have multiple swap file locations, set one for your flash and one for your HDD. My idea is that if the flash unexpectedly craps out, the computer will probably crash, but boot back up like normal because of the secondary swap file location. Anyway, wikipedia has a bit about how long flash should last. "most commercially available flash products are guaranteed to withstand 1 million programming cycles" I don't know how long a million cycles will last you, but it has been said that realistically that will be several years.
http://en.wikipedia.org/wiki/Flash_memory#Limitations
Just about forgot this, Vista will support this feature natively, and they'll be selling certified USB drives specifically for this purpose. It may be worth it to wait for the official stuff, then again it may only be a good idea for the paranoid, I really don't know how long flash will realistically last when used as an HDD.