It's pointless. The adapter will mess up air flow, hurting he fan's performance, and you really don't get much in the way of reduced noise. A bigger fan will push the same amount of air with less noise, but there's more to it when you need it over such a small space.
Axial fans push air out in a cone, that basically leaves a dead spot in the middle. Many heatsinks have taken advantage of this (it's all about the temperature gradient), or had designs that lessened the impact of not cooling the very center (the latter being more common).
The trouble is that the fan is trying to push out, but being forced to go back inward, causing resistance, and increases the static pressure, which is never that low on a HSF, anyway (part of why fans with good static pressure ratings, like Panaflos, do very well on HSFs). I'm not too knowledgable there, so there may be even more to it.
Getting a Japanaflo L, Nexus, Papst, low-speed NMB or Delta, etc., and maybe a [non-PWM] controller, are going to be much better options than a 120mm->80mm adapter.