The hens have to feel like sitting on them. It doesn't happen all the time. It also depends on the breed. Some breeds won't sit on eggs at all. They have to "go broody" first and you will be able to tell if they do that by putting your hand under a hen that is sitting on a nest. If they run away, they are not broody, if they puff up their feathers and try to peck your hand they are probably broody. What breeds do you have?
Leaving eggs in the nest will just make them more likely to get crushed, and if that happens, they will probably eat the egg. Once they eat it and like it, you will have a problem.
You might also wait until the eggs get a little bigger and buy or make an incubator to hatch the eggs.