For ours, we put foam in the windows, built window shutters, wrapped clear plastic over the screened windows (still allowing for ventilation), caulked the plexi-glass window, and made a ceiling from the clear plastic up to the exposed corrugated roof. I think if the birds are able to remove themselves from the elements of wind (draft), moisture, and frozen ground, they should be able to survive cold temperatures. I think that using your light as a heat source to not allow the birds coop to drop below 0 F degrees may also be an option, but think safety first. I know my son has some heat lamps for his lizard that don't get much hotter than a light bulb.
The fact that the nesting is frozen may suggest that the coop is one moist, due to climate? Two, the coop may never really be warmed up with the sun during the day time. Maybe you can do something to warm the coop during the day until your deep compost starts doing it's thing?
-Frozen Wings