There are all kinds of insects in the world, just because they are found in a chicken coop doesn't mean they will hurt the birds.
With warmer temps coming on they are hatching, often a bunch of insects just 'bloomed' don't last long...many only have a life cycle of a few days..
So no need to spray poison on your birds but might want to change out your nest bedding.
I examine my birds off the roost at night, after acclimating them to being touched at that time. I wear a dimmable tiltable headlight.
Google images of lice/mites and their eggs before the inspection so you'll know what you're looking for.
Part the feathers right down to the skin around vent, head/neck and under wings.
Best done well after dark with a strong flashlight/headlight, easier to 'catch' bird and also to check for the mites that live in structure and only come out at night to feed off roosting birds.
Wipe a white paper towel along the underside of roost to look for red smears(smashed well fed mites).
Good post about mite ID by Lady McCamley:
https://www.backyardchickens.com/threads/my-chicken-has-mites-now-what.1273674/page-2#post-20483008