The battery can be purchased as Kodak branded (usually with an external travel charger) or as a Fuji alone, and probably as 3rd party batteries as well, although I have not looked into the latter. I doubt Wal-Mart sells them, but a photo store will, or can at least order it for you.
By the way, pm me if you're interested in any accessories for a DC4800, since I've upgraded to a Canon G2. I've got memory, battery, charger, lens adapter, wide angle, telephoto, closeup among other things.