When I have one like that I cut off the shell debris and dribble some betadine or chlorhexidine on the umbilicus. Sometimes I'll give them a drop Baytril (antibiotic) orally.
Welcome to BYC! Can you post some pictures of the duckling? If he's not walking I suggest that you put him back in the incubator because he could get too cold or too hot in the brooder if he can't walk.