I don't think it will hurt them unless it is moldy, though there is a risk - because ducklings can do silly things - of them eating it and getting impacted. Just worst case that I can imagine. Some do, some don't.
Do you have any kind of fabric you can put over it?
You can use old towels, hay but it can get moldy quickly. I would consider old towels - they can be shaken out over the garden, rinsed, hung to dry and reused.
It took a long time and loads of patience with my Runners, but they (we) got through it.
I avoided cornering them, let them out of the brooder while I cleaned it (big brooder - I used to go in with them and it was fine, then one day it all changed and they were terrified). Then afterward we...