Yes, it could still be the PSU. Sorry if I was a bit confusing. What I was trying to point out was that this was a test that could help in finding the solution. It will give you an insight and head start. If the system powers on normally (no problems) with just the bare essentials then you will know for sure that it is the PSU. If it still exhibits the same problems when booting this way then it could still be the PSU, although the odds diminish a bit. If you want to give this test a good chance, make sure you unplug everything you don't need to boot: One HDD, CPU w/HSF, One stick of RAM, video card, keyboard and mouse. Probably have to hook up the floppy also. No other fans, optical drives, pci cards, HDD's; nothing at all but the bare essentials. Good idea to clear the CMOS before booting also. Do this again when you put stuff back.
As I said, if it boots normally this way, you will know the problem.