I think the danger is more from the mouse, rat, and wild bird droppings. Although, breathing in any dust is bad for your health.
I bought a painting overall to wear when I clean out the guinea coops. I put on a dust mask and safety glasses also. My DH says it is overkill, but I know that coop has droppings from rabbits, mice, wild birds (barn swallows nest in there), and assorted other vermin. The door to that coop is seldom closed and everything drops by for a snack.
I had coons steal a 5 gallon bucket that was 1/2 full of feed out of the shed. Gone, completely vanished. They must have dragged it through the field, across the road and into the swamp! Only other guess would be 2-footed vermin, I wouldn't be surprised to see that either.