Hmmmm, may I venture a frank opinion?
In your situation, with your reluctance and admission of being a softy, I'd rehome the roosters asap.
Most (nay I say almost all?) roosters at some point need a knowledgeable and firmer hand.
Children often become the target of a rooster attack as they are lower to the birds and can be seen as more of a threat to the birds. Children can become severely injured from rooster attacks.
So, I'd wait on having a rooster until your children are older and you are more experienced and confident around them.
My 2 cents.
LofMc