The complexity is to legalize it in a way that protects the sex workers from abuse by brothel owners, pimps, managers, agents, landlords, etc.
That said, what we have now, where it's a 100% black market, offers no protection for the workers at all and more or less opens all the doors to exploitation.
Not to mention likely results in an increase in STDs, abuse and rape
http://newsroom.ucla.edu/releases/decriminalizing-prostitution-linked-to-fewer-stds-and-rapes