Your ladies have decided that the nest box is a good place to sleep apparently. Chickens poop a huge amount in their sleep at night. Make sure the roost is comfortable and higher than the opening to the nest box, because they will often choose the highest place in the coop to roost. A good roost is a 2 x 4 with the flat (wide) side up, rather than a dowel, so they can sit on their feet. As someone else mentioned, blocking off the nest boxes either all the time until they are laying, or at night for a while to prevent them from getting in there to sleep will help. Another trick is to go into the coop in the evening and move them to the roost to get them used to sleeping there. Sleepy chickens are easy to move without disturbing them much if you keep the light level low (use a flashlight).
Poopy nest boxes are no fun, and that's a chore you should be able to avoid with a little investment in retraining now.