My current specs are in my rig, and as you well know, I was able to achieve stability at 3.36. I'll be suggesting parts along these lines, so bear with me.
First of all: Yes, the E6300 processor is exactly what you want.
CPU: E6300
Motherboard: Asus P5B or Gigabyte 965P-DS3 (the S3 model doesn't use all solid state capacitors, so it's cheaper if you want to go that route. The only different is lifespan.)
RAM: I'd definitely shell out for some DDR800 RAM if you can afford it. Don't skimp on this because it is very important if you want a stable overclock. I personally chose 2x512MB sticks of Corsair Value Select (PC5300 @ 667) because I was running low on funds. These sticks are working great for me, so I'm quick to suggest them for others.
Video Card: Not important. Pick the card you want. I've always had luck with nVidia cards (Linux is a little more fair to nVidia users), but it's up to you. If you do get an nVidia card, make sure it's made by eVGA, because of their "Step-up" program.
Hard Drive: Again, not terribly important. I have a 250GB SATA Seagate Barracuda and it's working perfectly for me.
Powersupply: If you're looking for both performance and silence, the Seasonic S-12 offers both. It has two or three 12v rails which help reduce voltage stability (I might be wrong here, so someone more experienced should feel free to correct me). Corsair PSU's are also very reliable. The Antec SmartPower 2.0 has had some issues in the past with overheating/melting/something or other, so I would recommend against it.
Cooling: While the stock cooler will still work for a mid/high overclock, I would recommend against it purely because of temps. They will be good enough for a stable/reliable system, but eventually you'll regret it and end up buying something better later. Your best bet is to just get an OEM E6300 and throw a Tuniq Tower on it.
Case: Depends on what you like. If you can afford it, a Lian Li will serve you well. Antec cases are also very sturdy. The Coolermaster Centurion 5 is a great case, and it's very cheap all things considered. Make sure to throw in a couple high CFM/low dBA fans. <22dBA should be perfect. The bigger the fan, the quieter it'll be. The Yate-Loons are a good buy.