Just my opinion, but I look for personality first, virility second and conformation third. We have a small space and the chickens, ducks and turkeys need to all share. Also, I find that it's much easier to move the birds and treat injuries and manage their housing if they are easy with being handled.
We ordered a breeder's choice group from Porter's 2 years ago because we had very little idea what we wanted beyond "not broad breasted". Out of that group we had 5 toms of 3 different breeds, Narragansett, Fall Fire, Sweetgrass. We originally decided to keep 2 toms, the Narragansett and one of the Sweetgrass. They were the 2 largest of the 5 and both were comfortable being picked up and handled. Also, they didn't fight with each other or our other birds. Both were gentle with the hens and were practicing mating and mounting (the Narragansett would mount anything left on the ground
). The other 3 toms didn't show any interest in mounting the hens or anything else. They didn't seem to have the "breeding instinct".
We then decided to only keep the Narragansett for breeding since the two toms were trying to mount the same hens at the same time. We didn't want the hens injured and didn't have the pens then to separate the toms so we just kept the Narragansett. His tendency to mount anything including the hens made us confident that he would prove fertile and we wouldn't need a back up tom.
Now this year we have more room, our original breeding tom and 3 jakes. 2 of the jakes have started exhibiting breeding behavior, one has not (they are all the same age). But one of the jakes that is trying to breed, is also very agressive and has attacked the other and drawn blood. He also doesn't like to be held or moved. So while he has the mating instinct and great growth rate and size, he will go to freezer camp.
The second jake that is displaying is too small to keep for breeding so he will go to freezer camp as well.
The third jake that is not displaying, doesn't seem to have any breeding instinct. He doesn't fight, but he also doesn't display or try to mount anything. He is the sweetest of the 3 and doesn't mind being picked up and moved. So while he has wonderful size, temperament and coloring, he may end up in freezer camp as well. We will give him a few extra months after the other two are processed to see if he is just slower to come into breeding.
So we may end up with just our original tom and see what we get from next year's poults. I would be worried if our breeding tom was slowing down, but he continues to mount the hens and anything on the ground so his longevity seems to be good.
So I guess my point in this long explanation is that you should watch how your birds interact with you and the other birds and pick the tom that will be a gentle but effective breeder. Most importantly, you need to wait to choose your tom until they come into their breeding instincts. Until that time, you won't know who is a fighter and who is a lover.
Then once you have a pool of good tempered toms to choose from, you can choose for coloring, growth rate, overall size, etc.
And any potential toms must show an interest in breeding. We have a friend who spend $150 on a beautiful pair of Buff Orpington chickens at a recent show. But the rooster doesn't know how to mount the hen, and has no mating instinct at all. He hasn't gotten a single fertile egg from him and is having to research AI. So you can have the most beautiful tom on the planet, but if he won't breed, he will only be a feed bill and won't do you any good.
Of course if you have the time and the space, you can choose just for size and color and not worry about temperament. You can keep each tom in a separate pen and introduce the hens to them as needed to get fertile eggs and not have to worry about your toms fighting. But I also think it's important to breed for temperament and well as conformation so I keep all my birds together and only keep the gentle ones.