i imagine it might be difficult to pry the axe head out of a skull sometimes, similar to how an axe head can get "stuck" in a log.
As far as ammo goes, most of the hardcore shooting type folks have reloading benches and enough supplies to reload at least 10K rounds. But, indeed, that wouldnt last forever!
For close range, id think maybe an ice pick or a claw hammer would work better than an axe, or, even just a smaller/lighter axe, hatchet or tomahawk.
For shorter range outside of arms reach, a handgun. I'd prefer a 38 revolver since they are simple and easy to maintain, but, most folks would likely prefer a 9mm since they can hold more rounds, and are faster to reload if you have extra magazines ready to go (though a revolver with speedloaders can be pretty quick, and they do make 7 and 8 shot revolevrs.)
medium range, im thinking pump action shotgun would be nice to have, you can bash em with it when you run out of ammo.
long range, im thinking a 22 rifle would have enough penetation to get through a skull, and be light and inexpensive enough to stockpile and carry a bunch with you.