The box is fine but if it's newer to them they're going to be wary of it. I don't force my girls to go in until they're already laying, and I only do it then if they can't seem to figure out where to go.
You said you had a plastic egg in it before and they were playing with it? That's completely normal. Just put it back in the nest. If they keep kicking it out, put it back. That's just the birds exploring the box. I use golf balls and they end up some weird places just before the new layers start laying.