Hey, first off, congrats on your first PC!
Second, I do believe that the Sandy Bridge (Intel 2nd Generation Core, aka what your i5-2400 is part of) -era Dell MT (mini-Tower) rigs, could take regular ATX PSUs, so you're in luck there, I believe that it can be replaced.
More RAM is good.
As far as GPUs go, what games do you want to play, and what is your monitor resolution and refresh rate (not size)? 1080P and 60Hz, you could even get by with a GTX 1650 Super, if you didn't need everything at "Ultra" settings. That's an entry-level 1080P-class GPU, only around $160 or so list price, and at least some of the GTX 1650 non-Super and "D6" (*GDDR6) models, may not need a PCI-E supplemental power connector (the 6-pin or 6+2-pin on the end of the GPU). If you put one of those (a GTX 1650 "D6" model) in, you might not even have to change the PSU out. Something to consider.
Edit: If you have a 1440P monitor, then consider a GTX 1660 Super or GTX 1660 ti or RTX 2060 or RTX 2060 Super. Those are all good 1440P cards. Also consider the Radeon RX 5700 and RX 5700 XT, those are AMD's 1440P-class cards. All of those 1440P-class cards, would require replacing the PSU, preferably with a 600-650W or greater model, with dual PCI-E 6+2-pin connectors.