From my experiance you will be better off without any roosters, yes I think sometimes they just stress the hens, and for me I tried 3 or 4 diffrient roosters, and every time I cull the rooster the hens became so happy and as soon as I bring another one after few weeks, my hens start to become stressed and they don't eat well.
Last tuesday I got red of the last and only rooster and eat him for dinner, since then I noticed that my hens are way more happy and they started to eat very good again.
I decided not to have any roosters, and if one of my hens goes broody I will just buy some eggs for her.
Omran