Two hsf's that come to mind for tight quarters are Tt's Silent Boost and the TMD version of Vantec's Aeroflow. I haven't tried the former yet, but I also don't yet have a tuner/ video capture adapter I will use, so I don't have all the parts ready for my own HTPC system, in a "real" HTPC-type horizontal box. Mine is going to have an XP-2800 in it.
Meanwhile, my (Socket A) gaming system has an XP 3000, 400 FSB, OC'd. The TMD Aeroflow wasn't quite up to handling that much heat (ran around 59-60 C at idle if the PC room got up above the "cool" temps I try to keep ambient at), so I swapped in a Gigabyte Neon 7, which is quieter, but takes up more real estate -- probably too much for an HTPC, but then the HTPC box doesn't have any window, and this gaming PC does (the Silent Boost is waiting its turn as a next project build here). That Gigabyte cooler looks very good on the 3000 (Royal Blue Neon color), so much so that I wanted another one for a newly windowed "Utility" PC here, and couldn't find it at a price I wanted to pay.
Another mid-sized cooler with a lighted up fan that I like the looks of, now that windowed PC boxes are available in my budget range, is Aerocool's Copper Cooler Extreme, quieter than a TMD Aeroflow (which isn't producing an objectionable sound level), but louder than the Silent Boost. And then you can get Aerocool and Aeroflow mixed as well -- I had to edit my second reference to the Vantec product above, where I'd misnamed it.
You might also want to check in at one or another of the web sites with HTPC subjects as the central reason for creating the sites, where they discuss the comparative efficiency, silence, and size of various hsf's all the time. There's one with an "AV" in the name that has a very busy message forum attached.
😉