That duct idea is good.
Have you used Peltier's before? If not, check out
Overclockers.com and their forums.
In short, with the amount of heat that Athlons put out at maximum, it is not likely that you will gain anything using a normal high-performance, air-cooled heatsink. The Peltier that could provide a temperature drop while passing the 70 watts an Athlon may generate would also generate something like 70 watts of additional heat. The heatsink will have to get rid of 140 watts, doubling the temperature rise across the heat sink. With the same heatsink minus the Peltier, the CPU will be cooler. That is untlil you get to a "super heatsink". With a "super" heatsink, the temperature rise across the heatsink ultimately can be low enough for the Peltier to be an advantage. A compact "super" heatsink would be water-cooled. An air-cooled super heatsink would be like 6 by 6 inches and use a high CFM fan.
A Peltier design which will do the job would consist of an array of 4 or more Peltiers with an addition power supply to handle the current. The Peltiers would be in a cold-plate/heat-spreader sandwich topped with a "super" heatsink.
Since you are not going to OC, Athlons will do just fine with a normal heatsink, such as the one that comes with the retail version. Or get an SlK800 with a slow 80mm fan. People claim they are almost silent, quieter than the power supply. Maybe with some kind of ducting rig you could use a bigger, slower fan and locate it outside the case.