I had to do this at a business - the motherboard, though it had a 3-pin connector for a power LED, would not POST if the LED was actually connected to that connector. Manufacturing flaw I guess.
We used a very small screwdriver to press the little latch down, and then slid the metal piece out of the connector, then back into the middle.
On the LED connector, on a flat side, you'll see a little square hole, through which some metal should be visible. That is where the tiny "latch" is - it's just a sliver of metal that's bent outward, keeping the metal contact from pulling out. Press that in gently (not too hard, else it will not latch when put back in) then slide the metal contact out by pulling the wire. That should do it then.