Word of advice be careful and on guard.
I used to live next to a neighbor when I was 12-16, who hated the fact that we had 6 dogs, legally licensed with the town, up to date on shots, well behaved, fenced to our property,...They used to come over all the time, there kid used to play with the dogs, and they kept to themselves.
Then one day they gave us hell, and made me a terrified little girl.
They threw poisoned meatballs into the yard, spied on me (I was home alone during the day while mom was at work) with cameras and binoculars, snuck along our property line...
They were great at first, and then all of a sudden (Just like you) they had a slight issue. Then over the years things like the above started to happen, the police were always called on us, and every day i was a shivering wreak hiding in the living room with a hand made spear Lol in case they decided to break in. (Paranoia of a teen, looking back I dont think they would have)
They were very pleasant when they got there way, but when we refused to just "Get rid" of our dogs and instead keep them quiet and try to be accommodating, that's when things started.
Exact same kind of man, exact same kind of situation (Only with different animals) and it was not until we moved that we ever got peace. Now I have my chickens and awesome neighbors who love our birds and the dogs.
I just hope this guy wont go to extremes to get his way like my old neighbor did

keep your guard up just to be safe!!
and best of luck to you!!