If you can post some photos that may be helpful.
Likely the dirt will come off on it's own.
If you have a lice problem, I would suggest that you treat your flock and their housing with a Permethrin based poultry spray or dust. Spinosad (Elector PSP) can also be used.
Sorry to say - DE is...