You can't connect and disconnect connectors with zero chance of damage (like USB? You've never seen USB get damaged?).
Taking into account that life is full of risks, and they non-zero, yes, VGA, DVI, HDMI, and DP are all fine for plugging in or removing hot.
What they are not designed for, is constant swapping around. If you're going to do that, get a KVM, or mount the connector to a stiff wall. All the torquing will cause failure over time (projectors, FI, typically have their connectors mounted to be supported by the case, while you're not going to get that with a PC).