Er, it doesn't go in a subfolder. When you do it right, you should have a C:/Program Files/Doom 3/ folder that has two folders in it: base and Nitro, and one .bat file (Nitro.bat). That's all. Nothing else. Then you can select the mod from inside Doom 3 through the mods tab, or you can run Nitro.bat, which boots Doom 3 with the Nitro mod already applied.
(WARNING: If you're using any custom weapon mods, like the Flamethrower 0.3 mod that's floating around, you have to copy that pak file into the Nitro folder or the game will crash if you try to give yourself the custom weapon. I'm not sure why, but it works fine if you just copy the pak file. *shrug*)