I have hatched some eggs that were somewhat dirty. As soon as the chicks hatch I take them and their egg shell out of the hatcher and put the chick in my brooder.
I think its safe to wash them if you use water that is about 105-110, use either an approved egg wash (I use the Brinsea product) or a dilute bleach solution (1 part bleach to 20 parts water), leave them in the water for no more than about 1 to 2 minutes, scrub gently if needed, and let them air dry well or dry on a soft towel, patting and not rubbing. Brinsea seems to suggest that its better NOT to rinse off their product, as it has some residual effect in killing microbes.