Not sure what impact it would have when flashing a ROM. I usually do factory resets between ROMs anyway, so other than having to re-encrypt the phone, I don't think it would be a problem. The boot loader, prompt to authenticate the encryption and does have access, so I imagine you could flash a ROM without having to do a factory reset.
I've not noticed any performance decrease, the I imagine there likely is a small impact.
Main con I have is that I also had to encrypt the SD card and now can't swap it out easily. So it's basically permanent storage now in the phone. But I didn't take it out much anyway, so not really a big con for me.