I agree with 11mini, although I'm not sure about the peanut butter. When I switched mine, they glared at it like it was going to attack them! Then one decided to go over and peck at that bright red thing with the shiny metal center, and water came out! She did it again. The others watched her carefully. Training time over.
We recently hatched out a lone chick under a broody and because of some extenuating circumstances, that chick ended up living in the house for a time. Even at just a week or so old she knew how to use the nipple waterer because that's what's out in the coop. I put a nipple into a container from powdered iced tea and she used it immediately. At first make sure they are drinking. You don't even have to be in the coop to know that - you can hear the sound of them tapping the metal center. And do take out their other water sources - it will only confuse them and make them ignore the nipples most of the time.
Good luck!