This is one of those topics where I'll start off with what you already know, which is you need to be comfortable with whatever you do.
Illness and disease can be caused by a number of things, and certainly manure has been associated with problems in a number of cases. And. Having a fecal test might be something to consider.
Hot composting is said to get rid of pathogens. Consider that idea. I spread duck straw on my gardens. I do not spread it anywhere close to harvest time for things that don't get cooked. While the nitrogen in duck manure is more likely to go away over time (water and temperature and microbes can turn it to N2 gas and away it goes), the phophorus and potassium in it should remain in the soil until plants take it up (unless you put it on a steep hillside, in which case it will go away with the soil when it rains).
So one option that I employ is that during mild days in winter I spread some raised beds with duck straw. That helps charge up the P and K in the soil. Cold and light - this is my opinion - take care of pathogens.
And by the way, the really ugly e. coli if I recall correctly, is not in every animal. I would think healthy ducks would have pretty healthy poop.