As long as you never leave them locked in there or they are not stuck in there because of weather when they are awake, you should be OK. The risk comes in where they get stuck in there for a period of time. How much time it would take for you to see problems, I can't tell you exactly. That depends on the personality of your individual chickens.
Adding more also reduces your flexibility. Right now, you could leave them locked in there for a while if you had a need, say retraining a hen to lay in the coop. Your risks of problems goes up if you try locking them up for a while. Last summer, somebody dropped two dogs off in the country for the good life. They killed several of my chickens. I could leave my survivers locked up while I dealt with the problem. I like the flexibility my extra space gives me.
You should be OK in the summer. My real concern would be a blowing snowstorm in Maryland in the winter. But you may have other outbuildings where they could get out of the weather if they can get to them. Last winter some of mine would go out and forage in the snow, but some would not. None of them like a blowing cold wind.
We all have different circumstances. As long as yours can get to the outside every day, you should be OK with another few chickens.