If the bugs are coming out at night to bite the birds (and those would be mites, not lice), you need to thoroughly spray the entire coop. Bleach will kill germs, but will not necesarily kill mites. Even though a coop is painted, the crevasses, cracks and corners can become bug hiding places. If you don't kill the bugs that bite your birds, they will continue to bite them. Yes, the ivermectin or eprinex or poultry dust may kill them, but until they die, they will continue biting. Not only do they rob the hens of nourishment, they also pass disease.
So, yes, treat the birds, but don't think that treating them means that you don't also need to treat the coop and roosts. If the roosts are over your head, I recommend using a step ladder.