You definitely need more pullets of less cockerels!! With four cockerels you would need quite a few more females though, so you might want to consider weeding out two of your cockerels. Most suggest about a 10:1 ratio of hens to cocks, although many get by with a smaller ratio. With four cockerels, I'd think you'd want at least 20 females (unless you pen some of the cockerels separately).
BUT, I'm guessing you're not interested in adding 18 more females. If you're talking about adding 3 or 4 birds, you could go with older birds, since the younger ones would outnumber them, kind of leveling the playing field. If you're talking more than that, I'd try to find a similar age. Be sure to quarantine...and integrating will take some patience and work...