How cold is to cold

Our 6, 3 Barred Rocks and 3 Golden Buffs will be 5 weeks old on Monday.
We have moved the coop to the unheated garage, covered it in clear plastic and zip tied an electric heater to the highest point inside.
The last time we checked it was 40 degrees inside the open part of the coop 3 hours after sunset. There is also a heat lamp hanging in the enclosed part.

These guys are running out of room in their box in the house... would it still be to cold for them to go into the coop?

The Barred Rocks seem to have feathered nicely and the Buffs are close behind.
I also plan on hanging one of my shop / grow lights above the coop so they get the light requirement they need. Hey...it starts my seeds in February, so it should help with the chickens I assume.
Thanks!