There is a saying that "Good fences make good neighbors." It sounds like this maybe the case here. i would make sure that it would not be convenient for the neighbor to wander around on your property looking for his wifes animals. If he hurts himself on your property then he sounds like a guy that would try suing you because he can, and he is a jerk.