This is what Metzer Farms says:
"Ducklings and goslings can be introduced to swimming water as early as one week of age but you must be very careful. They must be able to walk in and out of the water very easily. The water should not be too cold and they must be able to find their heat lamp for rewarming without difficulty. As they have no oil on their feathers at this age, they cannot be in the water for long periods or they will become waterlogged and chilled. Do not allow this to happen! But this exposure to water speeds the development of their oil gland and they can probably be swimming freely by five or six weeks of age."
http://www.metzerfarms.com/RearingInstructions.cfm