I don't understand how the birds could move the nest box, but....
I have 4 girls right now also. When they started laying, the one at the bottom of the pecking order wouldn't use the box--she layed on top of the box. (the boxes were tupperware totes with ventilation, total of 3 nesting areas). I set up a cat litter box on top so she could lay there. When 3 of the 4 hens stopped laying to molt, she resumed first, and she started to use the main boxes instead of the litter box on top of the nests. Now that all have resumed laying, she still uses the main boxes.
In other words, my observation is that pecking order can force one bird to not use the preferred box. So I suggest adding a box. It's not so much the 3 to 1 or 4 to 1 ratio of birds to boxes, as having a choice for the bird that doesn't seem to be "allowed" to use the main box.