I think it's partly due to oneupmanship on the part of the OEMs. There's a lot of competition between who has the best battery, and the battery technology itself has evolved at a remarkable rate.
This in itself brings problems, because Li-ion is a delicate technology, which needs a lot of care in charging: even relatively subtle changes, or small changes in capacity could mean that a new charger is required.
Some manufacturers have tried to standardise - e.g. Canon uses the same battery pack in virtually all of it's pro and 'high-end' consumer cameras. But I can't help but think that at least some of it is an attempt to control the market and keep prices up.