90% rubbing alcohol (avail at drug store) kills wasps, ants, roaches, etc. ON CONTACT. Pour it on the nest after dark. Should lnot hurt the chickns, and evaporates very quickly. You may have to do it more than one night to get the whole wasp nest.
With yellow jacket wasps, if you see where the nest is early enough in the spring, you can just kill the workers that are coming and going by standing there and spraying them one by one. There aren't that many early on, and then the nest dies out.
For carpenter bees, spray a dish soap solution on the bees themselves, and into the hole where the nest is. Also kills on contact. Then fill the hole with caulk.
I personally would not spray any organophosphate pesticide in my hen coop.