I keep the bucket out all day and bring it the house at night. But I also do a similar feeding as @HannahDuckLover and feed twice per day. I feed in the morning what they will clean up in about 30 minutes then about 2 hours before sunset I bring out their dinner that they should clean up in...
If there is a food source then there is always a possibility you will keep finding mice. You already know that. Ducks are very messy eaters. You already know that too.
Here's what I do.
I changed from a hanging feeder to a 1 gal bucket. They make a tiny mess but a whole lot less than before...