Absolutely for real wear a GOOD dust mask the whole time, and open as many windows/doors as possible. It is not about annoyance or discomfort, there are several serious health problems you can wind up with.
For the nestbox, I'd just clean it as well as possible (damp cloths laid on 'crusty...