Usually it is best to keep them out of the water until they are mostly feathered out -- 6 to 8 weeks . It is hard to wait though! And, if you are having that unusually hot weather then maybe it would be fine. I think mostly the concern is that they would get soaked and not come out when they should (because geese LOVE water) and then catch a chill. If daytime and night time temperatures are pretty high though, then I don't know if they would be chilled (I certainly wouldn't).
If you had a small kiddie pool, I'd think they could play in that for sure . . . I know my goslings were able to hop in and out of one by the time they were 4 weeks old . . .
I'd also worry a little about safety -- until they get their full size they would be more vulnerable to hawks/eagles and such on the pond . . .