How old are they? This sounds like behaviors between immature pullets and cockerels, not mature chickens. Are the girls laying eggs yet?
If you want to assure that the new boy is the father of the chicks you need to wait about 4 weeks from when the other boy mated with them to clear his sperm out of their system. Sounds like you have some time to wait anyway but a month may not be long enough for them to mature.
The girls have a part to play in this. The mating act is about dominance. If they are not old enough want their eggs fertilized the pullets are not ready to be dominated so they run away. Some may even fight back.
Once they become mature (usually when pullets start laying eggs) they expect a male to act like a respectable father to their potential chicks. Some boys mature a lot earlier than others. An immature brat has trouble meeting those requirements but usually when he matures the girls will accept him.
I may be totally wrong by assuming they are still immature juveniles so I can only guess but this sounds so typical of immature juveniles. I don't know how mature any of them are but if they are still young you just need to give them time.
