I have 2 120mm case fans, 1 120mm CPU fan, and a Asus 9600GT TOP HSF. I can confirm that the 9600GT HSF is the loudest part in my case. All my other fans (all 120mm) run at ~1000rpm and are virtually silent.
In fact, I found the noise generated by the graphic card HSF so annoying that I replaced it with a Zalman VF900-CU at low rpm. Now my computer makes no noise at all...
BTW, I absolutely hate the LED light of the VF900. It looks so cheap... I should have bought the non-LED one...