We have a barn on our property that was built in the 60s or so and had a serious problem with carpenter bees. We tried everything...swatting them, spraying them, spraying them with diesel fuel (that was recommended by local pest control folks!), nothing worked. I kept telling my husband 'if we fill up the holes, where they laid the eggs and some live, won't it kill them?' He said no for 4 yrs. Then last year we took some of that expanding foam caulking stuff, squirted it in the holes.... it cut down on our carpenter bees by about 90%. Cheap AND Effective.
To answer your question about them hurting your chicks... NO, so far as I know they don't even sting. I've chased them with tennis rackets and swatted them and never been stung. BUT they will destroy the wood in a fashion similar to termites, so get rid of 'em.
Good luck!