Well, the good news is, they aren't brown recluse spiders.
The bad news is, the spiders will still be around even if you torch your coop.
Spiders don't follow chickens, they just move into any likely accomodations. Black widows, like brown recluses, are relatively shy night hunters, so they will be up and about at work mostly when you are not there.
That's also why chickens are not such good eliminators of this kind of spider. Chickens happily eat many spiders, but chickens are mostly asleep when this spider type is active.
I can't recommend bug spray, but I do recommend knocking out their web structures. In my experience, a few days of having to rebuild everything usually convinces spiders to move elsewhere.
A dose of water is also an effective deterrent.
Then you will want to look out that they don't move on to your garage or your toolshed, for example!