Huh. It's been quite a while (aside from the recently-released X570 AM4 chipset boards), since motherboards had northbridge heatsinks with fans on them.
The last one, which was fairly popular, was the NVidia AM2 (AM2+ too?) NV430 + 6150 or something like that.
Yes, that fan should be spinning. It's probably dead, or is caked with dust (probably in the bearings too), or possibly could be saved with lubrication.
I would just replace it, though. Look up "Evercool Copper chipset heatsink fan". You should get some hits, if nothing else, look at the image links to see what you're looking for. Keep in mind that not all motherboards used the same chipset heatsink push-pin spacing, so you might have to jimmy whatever solution that you find, in order to get it to fit.
Something such as this:
THIS IS JUST AN EXAMPLE I FOUND, I'M NOT SAYING BUY THIS PARTICULAR ONE.
Fit for Most VGA. Design changeable fan cover, make the cooler can create the unique cooler depends on various users. Solid copper heat sink. 4 types clip size: 52, 54.8, 59, 79.8mm. Airflow: 6.89 CFM.
www.ebay.com
Sometimes, those copper chipset heatsinks, were also sold as "VGA coolers" (for those ancient VGA cards that didn't need a whole additional slot for a big heatpipe heatsink.
That said, maybe you could buy a dead really-old VGA card, and transplant the fan heatsink onto the chipset that you need.
Or just get a larger finned heatsink, like a RAM heatsink, and make sure that there is some airflow over it, somehow, and cool it semi-passively.