Not really with Linux. I packed mine with 32GB on the server and it really doesn't need more than 2GB at any given time. Though reboots to clear the cache reset the counters.
This will improve things a bit but, more importantly replacing the old HDD that would be prone to failure at this point would be the priority.
There's a massive difference between 32GB on a headless server and 4GB on a GUI desktop. Even if I found the most lightweight desktop OS that's still relevant
today, I would refuse to be stuck with 4GB of RAM. It's very hard to do tabbed browsing of the modern web without adequate RAM. The only exception is a mobile device, and even then, 4GB is considered poverty class today.
Anybody will tell you that an SSD is a huge improvement over spinning rust, not just in benchmarks, but in actual real world
feel. It's more than just "a bit" of improvement. One of the best upgrades I ever did was replacing a laptop's HDD with an SSD in 2013. And that was SATA 1.0.
But again, a 2011 PC is so old that I'd want to spend as little money as possible. Obviously used RAM and SSD that works reliably; and if somebody is giving away DDR3 when you pay just the postage, that is ideal. You can get a TEAM Group CX2 256GB for $18+tax on Newegg. That seems reasonable, but not a dollar more.
OP also floated the idea of buying a slimline caddy to hold a 500GB HDD; if he doesn't already own these parts, I think that's likely a waste of money unless (again) it's nearly free.
I agree with you that if this basket of goodies costs even $90 total, I'd rather roll that money into a modern laptop. It doesn't even have to be brand new; PCs depreciate so quickly you can likely get a great deal on a 2021 laptop.