This is an academic question since >99% of all speakers sold on the market either use passive crossovers.
Anyway, the difference is a passive crossover uses "zero" energy to operate, not including the resistive heating from the resistors, inductors and capacitors. An active crossover needs power to operate, hence active vs passive.
Passive crossovers are simpler to build and are often cheaper, and usually provide for a lower total cost of ownership overall once amplification comes into play. That's because active crossovers require one amp channel per crossed output (e.g. four amp channels for stereo high/low crossover), whereas a passive crossover requires only one amp for all the drivers it feeds.
Active crossovers are more difficult to build and are more complex, and are usually more expensive but can be less expensive in some situations. This happens when you need to cross over at a very low frequency, since the lower the frequency the greater inductance you need (for the low-pass). Inductors are big chunks of copper, so there's a point at which it starts to make financial sense to use an active crossover with a separate amp. It's easy to use a home theater receiver to split the LFE off the main L/R channels, but boy oh boy you'd never see one do it with a passive crossover.
Active crossovers typically are more flexible than passive crossovers so it eases the prototyping process, especially with the new digital units (miniDSP etc.). Active crossovers are much more powerful as well, in terms of what you can ultimately achieve. Dipole speakers, for example, require many more filters (crossover sections) than regular monopole speakers and therefore designing and building passive crossovers for them is comparatively much more expensive and time-consuming.
In terms of accuracy, you have fans of both, but personally I would trust op-amps, film resistors, and poly capacitors over imperfect power resistors, inductors, and sometimes even electrolytic capacitors for making sure that I get the response that I need.
Maximum power can be a limitation for passive crossovers in some situations, usually in pro audio. Resistors and inductors need to be sized for the expected amperage of the music signal, whereas since active crossovers use their own amps, there's nothing to worry about.
Sound quality I won't touch on.
Here's some more info:
http://sound.westhost.com/biamp-vs-passive.htm
http://sound.westhost.com/bi-amp.htm
http://sound.westhost.com/bi-amp2.htm
http://www.rane.com/note160.html
Sorry for the poor writing, I'm putting pen to paper as it all comes back to me.