The babies know who their mom is, but they feel quite safe with the Matron, the gander, the other goose and the older goslings. The mom looks a bit stressed sometimes when she wants the 5 of them to stick with her, but they just mix in. at feed time I separate the tiny ones so they get a good go at the crumb, but often they just want to be with the older ones anyway.
I´ve learned that whatever it is I want for the geese, they have their own ideas!

I´m sure Babe will be fine once he knows where his place is.
The second group of 2 males, 2 females that I have are actually made up of a couple of young males, a young female that didn´t successfully hatch anything, and has abandoned her nest now, and another young female with 4 goslings that I gave her to adopt after hers died by being too keen to keep it safe, if you remember!!!!! Well, she´s doing just great with her 4 now. Interestingly, the two males hang out together nearly all the time. and they keep good company with the female that has abandoned the nest. But, when I let the female out of her pen with her 4 goslings, the 2 males go potty over the goslings, looking after them, etc, and shoo the poor goose with no goslings away! Well, each time I let the goslings out and their mom, the other goose is getting closer and closer and sometimes she´s allowed to be part of the flock. Each day gets better. Geese are so interesting.
So, the reason for this rambling is, it´s quite possible that even though Babe may get a bit of a telling off by Sam, once they´re sorted, they may well be just fine.