One of the ganders will appoint himself as the leader and will drive the other ganders to the edge of the flock. They will hang out on the periphery but may not be driven away entirely. With lots of room, they might co-exist OK. I had a flock of almost ENTIRELY male Toulouse in Idaho. I bought four females and had six males (a year older) and SOMEHOW they managed to stay in a flock.
Maybe next year purchase a couple of female pilgrim goslings and when they are about a month old, you could let the adults raise them from there. Ganders are just as maternal as the geese (in my experience). My nasty lead gander Pinhead was especially nurturing and protective of those four little goslings females.
You may have to separate out a couple of the boys during breeding season, leave one gander with the goose and then bring in female goslings to be with the extra boys. I never could DOWNSIZE my flock very well ppffffft. So my solution is very biased.