I don't have a rooster since our city doesn't allow it but I read on BYC that aggressive roos will produce aggressive offspring. Perhaps the hens are also reacting to each other based on your roo's behavior. My chicks were a little older when I got them at 5 weeks but they were handled every day and hand fed treats. They're also at the bottom of the pecking order because we have two buff orpingtons who are 3x their size so I guess that keeps them in line too. They don't get bullied too badly because my orpingtons are very nice as well. Maybe just a peck and they get chased around for a bit.