At the cottage we used to pour a couple bottles of javex into the water supply to "treat" it as it was just pumped from the lake. We brushed our teeth and what not with it, I even drinked it a few times (thought it did not test very good. A fishy javexy kinda taste). Javex is fairly safe to drink in very small concentrations, it's pretty much what the city puts in the water supply. OR more specifically, chlorine. (I think javex is almost all chlorine)
I just washed my Brita with water and a clean rag though, to avoid any soapy taste.
You can also just go in a lake in a Darth Vader suit and wash it that way.