I'm sure you will get a few different answers. Personally, I would have no problem using the coop sweepings in the vegetable garden. The plants won't be producing fruit for several weeks/months so the nutrients will be absorbed over time. If the coop material is coarse, you can use it to make paths between the rows. Or use it as mulch.
Personally, I will compost the coop sweepings for a few months. If you saw the photos of what I tilled, that was the coop clean out from winter with chicken poo added daily. It is now rich, dark humus that I'll plant over the next couple of days.