A rooster is a pleasant sound, a long ways away. If your coop is close to another's house, it can be highly irritating. A home should be a place of security and peace, destroying that for a neighbor is not right. Don't wait for your neighbor to complain, go an talk to him about the possibility of this being a rooster, how you would like to try and keep him, but that you do realize that it is close to his house, and if it is a problem, you will take care of it.
Some roosters crow only occasionally, Some crow considerably more often. Some crow in the middle of the night.