Oh yeah, those are salt deposits. Fish can live in a range of pH and hardness and apparently goldfish are pretty tolerant so at least you have the right kind of fish
You don't want to change anything too fast because the fish need to adapt slowly. Replace 2 gallons of water every week for a while, then get into the habit of removing water when you vacuum out the crud in the gravel and adding fresh to fill the tank.
Also, if you don't have them, get test kits for pH, nitrates and hardness so you can see not only where the tank water is, but also where the tap water you are putting in the tank tests (it won't have nitrates, those come from the fish eating and pooping).
"
HARDNESS
The amount of dissolved mineral salts , namely calcium and magnesium, will determine the water's hardness. Water with high concentrations of salts is referred to as hard, while low levels would be indicative of soft water. Hardness is measured with the degrees of hardness scale (dH), which ranges from 0 to over 30 degrees, with 4 to 8 degrees reflecting soft water and 18 to 30 degrees reflecting hard water. This can also be expressed in parts per million- soft water is less than 75ppm and hard water is within 150ppm to 300ppm.
Goldfish prefer water hardness between 3 and 14 degrees of hardness. Even though goldfish can survive in water with a higher hardness, commercial kits are available at pet stores that will allow you to alter the water hardness, so there is no need to gamble with the health of your pet fish."
http://www.petgoldfish.net/aquarium-water.html