I can't comment on shirt colors but geese are very suspicious of any changes in their world. Mud boots, fresh bedding, a new feeder...they will make a racket about it.
The aggression you are witnessing is due in large part to being breeding season. Most ganders are jerks and very protective...