Never heard of a rotor that was "out of balance". I don't think it's physically possible. Well actually it is, on some of the very heavy truck rotors that have weights welded on to them - but I've never seen one come off.
You're experiencing lateral runout in your rotors - they're warped. If you are only experiencing a vibration while under braking this is most certainly the case, and is almost always in your front brakes. Take it back to the shop so they can put a dial indicator on the rotor and check it.
Unfortunately they'll probably just offer to machine the rotor back to true. This will take care of the problem in the short term, but the rotor WILL warp again. Once it bends, it wants to keep on bending - like a spring. By rights they should install a new rotor, and new set of pads as well, because the ones you have are already bedded into the set of rotors you have. They may be able to get away with just the rotor. In any event, I think it's unlikely they will do this free of charge anyway. It is possible that you warped the rotor after the fact (probably what they'll say) - did it do it right from when you picked up from the shop? Try to take your vehicle back right away after any repairs if something isn't right.
FYI, an easy way to warp rotors is to come to a hard stop from highway speeds and then sit in one spot for a while (like a stoplight). What happens is your rotors cool off quickly by radiating the heat away - except for the part which is covered by your brake pads. Now the rotor is cooling unevenly and it may warp. What I do in situations like that (while waiting at the light) is let the car creep forward a few feet slowly, so the whole rotor has a chance to cool at the same rate. It only takes a few seconds, until that initial heat that's built up can radiate off.