Disconnect everything but the mainboard and see if it happens. See if the fan is turning fast or slow or not at all. If you stop the fan by sticking something into it (preferably just slowing down the hub in the center, not jammed into the blades), does it go away? (Don't do that for more than a second or two by the way.)
If it still makes the noise with only the mainboard connected, unplug it completely from everything except the AC power cable. Use a bent paperclip on the big ATX power connector that goes to the mainboard, connect the ONE green pin to any black pin. The green pin is the one used to power on the PSU, you're just acting like a power switch. If it still makes the noise, return it. Even if it's not really harmful, it's annoying and a defect.
It could easily be a defective component.