I had always thought of vigor as a long-term thing in a flock of poultry ... unless one is breeding for the "hybrid vigor" of a meat animal, in which case you're likely just looking for early size compared to the standard (parent) breeds, so are not concerned about long-term health or survival skills or breeding potential. As in Cornish Cross ... But I don't think that's the kind of vigor we should over value in a breeding program.
I think the pecking order of a flock of birds is a sum of all the parts. I have seen birds get to the bottom of the pecking order and then start to show health problems, but I've also seen previously "low status" hens be great broody moms and gain some flock status. I also realize a LOT is going on with the flock that I don't get to witness, so there is much I don't understand. I don't have a ton of experience, I'm only in my second breeding season now.
I've seen some interesting interpretations of the word "vigor" lately, so I've been thinking about it a bit.
It is long term and short term. You cannot have it later if you do not have it now. It is something to be maintained and preserved.
Neglecting this point is to run the birds into the ground.
There is always someone on the bottom concerning the pecking order. It does affect behavior and the reason I used it in the response to the OP. That the OP should caution against these types of selections, and focus on characteristics that are easier to quantify.
Back to the OP (that never brought up pecking order), if you remove your top two cockerels, suddenly the third will be bolder and more assertive. The behavior will change because the dynamics have changed.