PC2100 memory is designed to run at a 133MHz double-pumped bus (Double Data Rate 266MHz or DDR266). PC3500 can also run at the DDR266 speed, but is capable of being operated much faster (up to DDR433 guaranteed and probably beyond). At DDR266 speeds, if all the other timings are the same too, the PC3500 doesn't actually perform any differently than the PC2100.
Regarding those "other timings," take Column Address Strobe Latency as an example. CAS Latency is sometimes further abbreviated to CL, so you will see CL2 or CL2.5 commonly mentioned. 2 has a quicker response, just like a hard drive with a lower seek time is going to have faster average performance. Where a PC2100 module might be rated for an overall 2.5-3-3-6 set of timings, the PC3500 might be able to run (on an A7N8X, anyway) at 1-1-1-4 (and for the skeptics who've never seen a 1 option, yes, they're in there).
If you took the two modules and raised the speed to 166MHz (DDR333) then the PC2100 module might make it (I have a couple that do) but chances are you would have to add some slack to the timings, while the PC3500 isn't even breaking a sweat. Obviously, good-quality PC2700 is a good compromise if you don't anticipate pushing things a lot further than this. The PC3500 may still offer tighter timings that the PC2700 can't handle, but the law of diminishing returns is setting in.
Where PC3500 would make a lot of sense is if you want to boost the "pipe" between the CPU, northbridge and memory as much as possible by overclocking the CPU and memory buses synchronously to 200MHz (DDR400) or higher. Wait, you say, the CPU isn't going to let me go from 133MHz bus to over 200MHz bus, that would be overclocking it 50%+! :Q Yeah, but with nForce2 and a Thoroughbred, you can begin lowering the CPU's multiplier to keep it within its comfort zone. The article at Hexus illustrates very clearly that using this technique to boost a 2400+ from 133MHz to 166MHz resulted in about a 5-7% performance boost even without raising the total CPU MHz from its stock 2.0GHz. So with DDR433, or PC3500, you could conceivably keep the CPU within 80MHz of its intended speed while supercharging its memory bus to the 220MHz (DDR440) level, provided everything else in the system can hack it.
The question is whether today's nForce2 boards themselves can handle that kind of speed. Jury seems to be out on this question. Some people are going up to 210MHz and higher, I saw a 225 reported, but Evan found that the boards in his roundup were peaking around 195. I picked PC3200 CL2 from Corsair in the end.