I didn't say put insulation on your wall. I ask how we'll insulated is your coop? If you have no insulation at all that will directly contribute to how cold your coop will be. I use a heat lamp with a 100 watt bulb. My coop is ventilated and it stays above freezing inside.
If you think they will freeze I would shut them up. Not sure how cold we are talking about but if it's below freezing point for several days or weeks they could freeze or get severe frostbite. Do you have maybe a heat lamp you could put in your coop just to keep it a little warmer for them...