That's perfectly safe. 60C and up is where it starts getting a little hot. I wouldn't really recommend much overclocking with the temps you have, especially if you need to raise the core voltage. It would probably be wise to stay below 55C when overclocking.
BTW, what kind of Sempron is that. I just got one of the 90nm 2800+ for my bro and it doesn't break 35C under full load. I guess your CPU is of the Socket A variety.