I always suggest you keep as few roosters as you can and still meet your goals. Roosters don’t lay eggs so the perfect number for you may be zero. The only reason you need a rooster is if you want fertile eggs. Anything else is personal preference. However you may have a situation where a rooster could still come in handy.
All chicken breeds were developed by inbreeding. Practically all Grand Champion show chickens are created by inbreeding. Inbreeding can bring out the bad genes which may make it easier to get the bad genes out of your flock if you have any. It also gives you chance to strengthen the good genes if you select which chickens you want to breed with care.
A standard model for small farms for a few thousand years has been to keep roosters and hens from your flock for a few generations, maybe four or five generations, then bring in a new rooster to bring the genetic diversity back up. Then go through the cycle again. There are other techniques you can use, such as spiral breeding or pen breeding, but this is the simplest for a small backyard flock. You do need to make sure any defective chickens do not breed though.
There is no magic number on hen to rooster ratio that solves the potential problems of having multiple roosters. Many breeders keep one rooster with one or two hens the entire breeding season without problems. Many people keep multiple roosters in the same flock. One huge key is how much room they have. If you have a lot of room, it is possible the two roosters and two hens can work things out, but you are probably going to see a lot of drama. Then it sounds like they will interact with the neighbor’s chickens unless you keep them confined. I have no idea how that will work out.
Personally I would never try to keep two roosters and two hens no matter how much room I had, though there are a few people on this forum that say they do successfully. I don’t know if one or none is the right answer for you. Your neighbor obviously has a rooster. If they are close enough to interact like I think they will, your eggs will probably be fertile whether you keep a rooster or not. That’s probably fertile by their daddy which again is not usually a big deal. If you keep one rooster, he will likely keep his flock in his territory and separate from the other rooster. If you only keep two hens and no rooster, they may continue to stay with you and lay eggs for you or they may join the other flock. There is a good chance they will join the other flock during the day, but will they come back to you to lay eggs? Hard to say.
