It depends on what you do with it.
For a compile machine, 4 cores will rock assuming the I/O system is up to snuff. This is because compiler processes are generally independent of one another so your throughput will scale roughly linearly until your I/O system can no longer keep up.
Similarly, I have a SMP system that I use to perform Wall Street analysis. The individual parameter sets are independent of one another so they scale linearly with the number of threads/CPUs.
I expect memory might start to become a bottleneck with 4- or 8-core CPUs though.