The conceptual problem people have is that they want to "upgrade" in the abstract. If you settle in your own mind exactly what the deficiency is, you can frame an upgrade that makes sense. If there are no upgrades that make sense, the only thing to do is live with what you have, or move on. Except for the HD, what you have in the emachines seems usable as is. In your own mind, get it straight what you want to accomplish. Tell people here, and get a reasonable reply.
Looking at the little emachines I'm fooling with, I'd say it is nicely designed and engineered. But they saved everywhere they possible could without considering the reusablility of the components. Every retail 810 mobo I ever saw would take up to 512M. This one is specified at 256M, with 128 DIMMs the largest permissable. It has a single external 5 1/4 inch bay. The cooling for the case is OK only for the type of CPU they chose. ( I see in the stores that the present emachines have a separate fan for cooling.) The PS hovers close to the mobo right over the CPU. This is actually a good design (and one that Intel and AMD prefer) because it picks up heat from the CPU HS cooler; Except the clearance is too low for the bigger HSs that are now used
If you upgrade into that case, you are locked into mobos of that era, unless you are willing to take a risk. If you choose an old mobo, you are limiting the performance of the possible upgrades to something like what you have already.
A reasonable upgrade would be a HD. You can always reuse a HD if you abandon the upgraded emachines.
Maybe a CPU. If the mobo will take a Coppermine CPUs, you can go up to 1G, but you may have to use the old HS to fit.
Since that mobo you tried did not fit, did emachines use a true microATX mobo? What are the dimensions?
A new Athlon motherboard may not work well, or at all in the old case. If not, you have to get a case to go in that direction. That case Mechbgon mentioned is not too expensive on the Internet (or its more modest little brothers SLK2600,SLK1600.). Although all you really need is any kind of physical ATX case, and the PS is the important part, combos like the Antecs are a better deal pricewise. Once you get a case with a proper PS, you can upgrade asynchonously, getting just what you need, when you need it. (You could probably put the emachines mobo in the new case, if you wanted, until you build up more cash to go on.) It is easier when you don't have to take the total price hit of everything new. If you don't ever upgrade, prebuilt machines are a better deal.
Saving the memory? There are rebated sales for 256M DDR every week for $20 and less.