I would guess your problem stems from having so many males and only one female, plus a very small group on to of that. They do best in larger groups and need at least a 1:1 ratio male to female. How old are they?
The common guineas you see are all the same; different colors aren't different...