Other than you saying something is sticking out of it, it sounds a bit like the oil gland. When ducklings get access to water sooner and more often, their glands will start to develop faster than those ducklings who don't. Plus some just develop at different rates. I would also like to see a picture.
If you are giving chick feed make sure you are also adding a source of niacin. They also need grit if you are giving them food other than commercial poultry feed.