You can sand the rust off and solder on a new piece to cover the hole. HOWEVER, when you heat galvanized steel it releases substances that are very, very harmful if you breath them, so wear an appropriate respirator. If is easy to solder though. I have built, or repaired many a roof jack that way (mostly for evaporative coolers).
Then paint it. Once cured there is nothing to leach, just watch out for paint chips (although they are pretty benign) as it indicates further corrosion.
I must say that I HATE galvanized fountains though, I went with a five gallon bucket and poultry nipples, I actually cut up the galvanized waterer to make flashing for my roof. Worked great for that.