The problem is the calcium content. Laying hens need extra calcium for the egg shells so Layer has about 4% calcium. Studies have shown that if growing chicks regularly eat that level of calcium they can develop internal problems with their liver and kidneys. One bite won't kill them. It takes a steady diet of feed that rich in calcium to possibly cause problems. It depends on how many total grams of calcium they eat in a day over several days. If they forage for a lot of their food as they often do with a broody hen they probably won't eat enough calcium from Layer feed to harm themselves. It's not always a real simple answer.
Not every chick that eats that much calcium will have problems but the studies have consistently shown that several will. When they do these studies they don't just see how many die or compare weight gain with other chicks fed a low calcium diet, they also cut some of the chicks open to see what is happening to their organs. Often you get chickens that don't show any symptoms but they just don't thrive and produce like you want.
In an emergency feeding them Layer for a couple of days won't cause any problems. But what you want is a feed that has about 1% calcium so they get enough to support good bone growth but not enough to cause problems.