First thing, blow them out with compressed air - otherwise the accumulated dirt/dust acts as an abrasive and can scratch the optics.
There are some mini-sponge-tipped applicators (q-tip thingers) made for getting into fiber optic sockets. There are also some formulas of cleaners just for fiber optics. Since this is not a production box, you can probably use high-content alcohol (90% isopropl or grain alcohol - "EverClear" - something ~200 proof). While you're cleaning, you can make some Christmas punch

).
Generally, when cleaning optics, you only use the cleaning surface once (so that the trapped dirt/debris won't scratch). Usually you start in the middle of the lens, then spiral outward to the edge. Then toss that applicator and do it again, repeating until the the entire surface has been swept, and the swabs come up clean. Don't drench the swabs - dip 'em & flick 'em to lose the excess.
That's about it. Personally, I'd like to offer congratulations / a pat on your back: most people clean the bejeezus out of the fiber ends, and totally forget the socket that's been sitting exposed to air for the last umpty-squat years.....(plug unused sockets ... cleaning 'em is a PITA).
Good Luck
Scott