First of all, you can never go by what someone selling a coop says the coop will hold...they always overstate. A good rule of thumb is 4 square feet per bird in the coop plus 10 square feet per bird in the run.
Chickens actually have a quite complex social order. What you've done is forced two groups of birds that are strangers to each other to suddenly live together. Of course they're unsettled. You'd be pretty unsettled if a family of strangers moved into your house all of a sudden, too! Usually what's recommended is that you quarantine new birds (house them separately from your existing flock) for a month, so you can watch for any diseases/parasites. After that month, you pen the new birds next to the established ones so they can come to recognize them without being able to attack them. This "getting to know you" can go on for a week or two. If all seems OK, then you integrate into the same pen, watching carefully for problems.
The new hen who's not laying may take a while to get used to her new situation. Stress depresses laying, and it's very common for hens to stop laying for a while when they move to a new home.
The other factor here is the juvenile birds. You don't mention how old they are, but usually older hens will be very vindictive towards younger ones. Usually it's recommended not even to mix two different age groups until the younger birds are almost as big as the adult hens...close to being fully grown, in other words. Some of the older birds are probably preventing the younger hens from even going into the coop. Do watch to make sure that everyone is able to eat and drink. Sometimes established birds will prevent new ones from getting to food and water.
Pooping in the nestbox is usually solved by making sure the roost is higher than the nestboxes. If that doesn't work, you can block the nestboxes off at night.
