Actually, there are microCD-MP3 units that are even smaller than the iRiver... given the smaller CD size (same size as those business card CDs) it still has more storage ability that an MD, similar size, and better sound/fidelity depending on how you encode your MP3s. (MDs cut out all sound above a certain frequency, so they may sound better than the "average" mp3, but not better than a well-encoded one)
Plus, you don't have to buy into $ony's encoding technology and media tech. Although microCDs might be a pain to get...
The main advantage of MDs though is the builtin recording ability... but if you don't care for that, a small HD MP3 unit or microCD MP3 unit is best for portability. Personally, I think it's a pain to put an MD player in your pocket, it's still to heavy and I don't like it either squeezing it into a tight pocket or slapping my legs in a loose one. Belt clip is fine for me... something like the Archos Jukebox might be something to look into.
You could consider a CompactFlash or one of the old SmartMedia MP3 players for ultimate portability... SD is better than SmartMedia and hopefully even if they introduce more security measures into SD, you could still use your MP3 player. I was in Japan last year when they first had SD MP3 players in stores, a tiny watchlike thing with 64Meg cards. Thing about buying stuff in Japan, if you're not getting some thing early or unique, isn't worth it since we get stuff pretty cheap here in the US... even with the weakness of the yen.