I can't offer much in the way of advice but I can sympathize. We had a neighbor like that and I hate confrontations as much as you do. Our neighbor called the township a couple times to complain about our yard being a "mess". The thing was, we were in a rural community and "mess" was not spreading chemicals on our lawn to suppress weeds or mowing twice a week. They kept their yard like a subdivision but the nearest subdivision was miles away. Luckily the township came out and shook their heads at the neighbors. We were doing just fine for where we lived. But the neighbor kept up with her nastiness, complaining about us behind the fence LOUDLY when she had company. I would suffer from anxiety attacks when she started up. We moved to an even more rural area but haven't sold the house yet. I got a call from our realtor the other day saying that the neighbor called her with a threat - that if we didn't mow the lawn she'd call the township -_- Not a request or a concern, a threat. But the township, once again, is on our side and says we only have to mow once a month and we have it mowed every other week!
Anyway, if I had any advice at all, it would be to contact your township and give them a head's up about the issue and see what your rights are and what his rights are. If they're supportive at all, they'll help you out. We put up a privacy fence to get rid of that lady and it ticked her off big time but it worked pretty well because she couldn't really complain after that... except for the grass which is her problem.