You are wishing. Wishing they would all be nice and get along. The problem as I see it, is the young boys are coming into their own, and giving that old feller some competition. The old boy is not in favor of this, hence the fighting. And fighting is rather unpredictable, it can solve itself, it can be a fight, wait, fight again, or it can get real ugly and fight to the death.
With your heart on your sleeve, you are not going to like the last option, and really I don't think you like the situation that you have now, and to be honest, all this fighting adds to the stress of your flock and is probably upsetting your hens. If you leave it as is, it is probably get worse, and maybe get better the older they get, but it could get pretty violent.
I also think that if you pull the old boy, the younger boys are going to start fighting. Being raised together does not really help with roosters. I am recommending you let go of 4 of these boys, and keep the old flock master and one boy. That might give you some peace in the flock. I always solve for peace in the flock, I don't like fighting birds, and really what is the difference if you have fighting birds in a ring and people paying to watch verses fighting birds in a flock and no one watches.
Now in the years to come, you probably can let more roosters be in your set up. You do have space. And with well established multi-generational flock, they develop a true chicken society and work out multiple males. However this really develops over years, hence people with long established flocks seldom have trouble with roosters.
I know this is not what you want to hear. But really roosters are where the romance of chickens meets reality of intact males as AARt is fond of saying. Good husbandry is about taking good care of the animals you have and making decision for the flock.
Mrs K