If you have space above the next boxes, put a cardboard box or better still two - like a shoe box - up there with one egg in it (or a fake one or a golf ball).
When you hear the 'I'm just popping off to lay an egg now' clucking noise. Get there before the egg is laid and put a girl up into the cardboard box - as it's higher she's likely to be quite pleased and will keep going there, do the same with another one. The ones you don't lift up probably will never bother going up to look and you might just get them to accept that a box each is a good idea.
If you don't have space above, try just sitting a shoe box on the floor, with an egg or golf ball in it. Same thing though, put the girl in so she knows that's an OK place to lay.
Ours nest in all sorts of things, old hats, buckets, the wood shed, an old box, there's one on a shelf on our terrace and two in the garage but they all get moved up to the coop before hatching.