I see two choices:
1. Ask him nicely to move the garden and hope he complies, if he doesn't then either deal with it or skip to step 2.
2. The land has to be owned by someone. If it is easement then there would be rules about what you can and can't do on it. A few phone calls will probably solve the problem. But then you might make an enemy etc etc.
I'd start by looking up the rules that govern the land behind your house. You might even be able to put a fence up right to the water line at which point his garden becomes within your fence.
BTW if there is an an HOA then just call them and they should solve it fast. In our HOA easements are public areas and you can't do anything on them at all.
Unless the guy is a total ass he will probably comply with your request rather then get in a long drawn out fight that he should lose in the long run.
		
		
	 
I am pretty sure with the covenants we can't build a fence and even if we did it wouldn't go through his garden since it is on an out lot and not our property, plus it would block the view of the nice part of the lot ( the established out lot with wild life).
The guy is smart obviously since he is working around the rules and knows the people who schedule things like mowing the out lots.
For example, we got a bill for 35 bucks to 'maintain' all the out lots (mowing), but every home that I see mows up to the retention ponds so the mower guy isn't going to mow them really...he is mowing a couple of ditches and that is it...which totals about 3,000 dollars (seemed a bit high to me).
The end game after I sit here and type this is at the end of the year ask him nicely to move it to his side of the lot and see what he says...he if pushes back I can obviously escalate it and complain but I don't want to do that...but I will.
I will live with it for this year and see what happens at the end of summer.
He seems like a stubborn kind of guy set in his ways so it will be interesting.