I knock the nests down too, not always at night but I’m not allergic to the stings. People that are have to be more careful. I don’t find the wasps dangerous to the chickens, but I don’t like fooling around in there when wasps are around. They can hurt me!
A quick story. When we had 2 week old chicks in the brooder in the coop, my wife stunned a red wasp and dropped it in the brooder. It was a blast watching the chicks play keep away until they finally ate it. They pecked it to pieces to get sizes they could swallow.
What a sight to see!
I tried giving a stomped yellow jacket to the chickens since I had read they liked to eat them. They came up to it, looked at it carefully, and BACKED away with great trepidation as if to say, "I have been stung by that thing before and don't want to eat it!" Just imagine FOUR or so hens all backing up at once!
They make a bug vacuum that I tried but the battery didn't last long. It was supposed to give a shock to kill the bugs. I couldn't recommend it unless you get a durable one.