Before you do any electrical stuff, shut off the power for that room at the circuit box. Then turn on the wall switch to confirm that the power is cut off.
Only thing I can think of is to take down the fan fixture, disconnect the wiring, then put it back. Make sure the loose wires are capped. The plastic cover can be removed by yanking it down a bit, then you have to reach behind the cover and squeeze a couple of fasteners to take the cover off.
Maybe someone else has a better idea. I recently replaced a bathroom fan--working in the attic was hot as hell!