No, I don't think you should be worried. I subscribe to the completely opposite school of thought, just to confuse you more!
But I feel that history and biology strongly support it. My compost pile is rather designed to be a part of my chicken run, and everything we throw out from the kitchen or fridge or gardens goes onto it, moldy or whatever. The chickens eat whatever they want to eat out of it, and the rest composts. They also eat the bugs and worms that live in the composting material, and their scratching as they forage naturally shreds and turns the materials to some degree even. If you search around for "chickens and compost" or "compost runs" etc. you'll find all kinds of people doing similar things. There is even a commercial composting company in northern VT that raises layers that forage in the piles. Chickens and compost piles have a natural symbiotic affinity that can be utilized to great success with a little planning.
Chickens love to eat garbage. Just the other day I saw a wild one eating something nasty-looking that someone had dropped in a parking lot, and it looked quite healthy to me! The Icelanders call their native landrace "pile chickens" because they are always seen foraging in piles of rotting manure and such. Chickens have been recycling food wastes and making use of garbage heaps since the dawn of time, and it's what makes them so functional and frugal. Compare also pigs.
As to safety concerns, I have yet to see a chicken eat something overtly harmful to it. I have never had a chicken die of botulism poisioning, nor do I know anyone who has. They are very good at knowing what is ok to eat, as evolution and millenia of sorting through garbage for edibles has conditioned them well. I'm not saying that weird things couldn't happen--but I just cannot in good in good concience recommend precautions for things I don't think are worth worrying about. You could purchase and install a lightning rod on your coop to protect your chickens from random strikes, or stock up on tiger traps, in case there is an breakout at the local zoo, or not buy storebought feed because it might be contaminated with rat fleas carrying bubonic plague, but I don't see the point of that kind of thinking.
So, in short, I don't think you should be worried in the least. And consider allowing your chickens regular access to the compost pile, if it's practical for you to do so. Just like I tell anyone who takes my chicken-keeping workshops.
Cheers!