Hey Mosslack, it certainly can't hurt to give it a shot. First I'd try the distros you have, and failing that, try the latest iPC or iDeneb on it.
I build a ton of computers for clients, friends, family, etc. and whenever I'm not up against a hard deadline for delivery, I always try 'Hackintoshing' odd hardware just to see what happens, and its surprising how often it works. Not long ago, I had a dual Xeon 771 SuperMicro server installed and working amazingly well with 10.5.5; it was probably a match for the real dual 771 Xeon MacPro had it been mine to really mess with. Of course right after Hackintoshing it, I had to nuke the install and put Windows Server on it for the client. It's odd- I've seen hardware that seemed perfect on paper completely fail, but then weird oddball hardware that I thought "not a chance" works fine. Heck, my first Hackintosh was a really oddball socket 939 AMD running OSX 10.5.2. It still amazes me how well that system worked for such odd 'Mac' hardware.
As long as the drive controller is supported, OSX will run on just about anything. The problem as I'm sure you know, is all the system components. You might find that OSX runs great on that hardware, but then something like the LAN won't work. That'd be a real deal killer for a server unless you can use a PCI card- not always easy if it's a slim height chassis. Find out what the board components are and if there are known kexts for them.
Also- how well does the non server version of OSX work as a server? Is your G4 flakiness recent, just a software thing, or has it never worked all that great as a server? If Hackintoshing the new server doesn't work, I'd highly recommend clarkconnect linux. I've got a NAS box running that that's been going strong for years, serving up files to both Macs and PC without fail, with nearly zero downtime.