Safes would cost you more than all the fees banks have ever thunked of.
Anyway, I'd move the money out if it looked like the bank was headed down. I abandoned my credit union when they started loaning in the commercial real estate market. They didn't go under but their savings interest rates tanked below that of even the national banks as they tried to save their own butts.