What are your goals for keeping a rooster? Are you planning on hatching chicks? If you are not, then why do you want one? You need to know your goals so you have a better chance of getting what you want. Often, especially with someone just starting out, that's not easy.
If you are planning to hatch chicks what do you want those chicks to be like? If it is important to you that they look like the breed standard then get a copy of your Canadian breed standard and see which boy comes closest to that. If you want to raise the chicks for meat then look at the boy that gets bigger at the age you want to butcher them. Roosters don't lay eggs so it's pretty hard to use egg laying as a criteria unless you know how well their mothers or grandmothers laid eggs..
Behaviors are a lot harder to judge. Right now you don't have hens and roosters, you have immature pullets and cockerels. Behaviors can change a lot as they mature. Watching them go through puberty can be rough because there is often violence involved, between the boys and how they act with the girls. Typically when the boys and girls all mature things calm down a bunch but it can be wild until they do. They don't all mature at the same rate. Another big problem is that the more dominant boys can suppress behaviors of the less dominant, which often means less mature boys. If you remove a dominant boy the one remaining can flip in behavior. Trying to judge how a cockerel will behave when he is both immature and being held in check by a more dominant male, either mature rooster or immature cockerel, can be extremely challenging. I don't always get it right.
I don't know how much room you have either, inside or out. Typically the more room you have the better. One thing that can happen with immature cockerels is that their hormones are driving them crazy to mate with the girls but the immature girls don't want to have anything to do with them. Sometimes the girls will stay in the coop and maybe even on the roosts to avoid those rowdy boys. Even if you have a lot of room outside that can happen but the less room you have often bad behaviors are magnified. I have a lot of room outside and this hardly ever happens, but it can. It's a little unusual for those other boys to stay inside like that, but that probably means they are slow to mature.
In other words, what you are describing doesn't sound all that unusual to me.
I know what my goals are, what I want to chicks to be like. I typically have 15 to 25 cockerels a year to choose from. One of my goals is playing with genetics so I typically keep a new rooster every year. I start by butchering the ones that obviously don't meet my goals, take the easy ones first to narrow it down. It's usually pretty easy to get down to the last 2 or 3. Then it can get hard, but at the same time all of them might be a pretty good choice. As I said I still sometimes get it wrong as far as behaviors because behaviors is one of my criteria. With more than one boy certain behaviors can be suppressed.
One of my goals is early maturity. Part of that is they put on more meat earlier, which is better for my meat goals. But I think it helps with behaviors too. At the end of the day the rooster has to be the flock master for the flock to function a it should. The hens have to respect the rooster. A weak willed rooster with no self-confidence often has trouble managing that. Instead of winning them over by the sheer strength of his personality and magnificence he has to rely more on brute force. I find the transition to a peaceful flock can be easier with a self-confident boy (which generally means early maturing) but getting through puberty can still be pretty rough.
One he matures a good rooster welcomes the girls outside. Reducing the number of boys can help that process a lot too. But whether you have one or eight boys, getting through puberty can be rough to watch. As long as no one is getting hurt I don't worry about it but violence is often involved, either between the boys or wit the girls. You do need to watch for injury.