Have you tried actually talking to the neighbors and asking what the actual problem is? There may be a way to mitigate the problem easily but you'll never know if you don't ask. Communication is easier than feuding and you may even get some help if fencing turns out to be the answer.