With those temps, they DON'T need a heat lamp. If your coop is secure, they could go out with those temps if you give them a wool hen or a huddle box.
Chicks brooded in the house should be off heat by the time they are 3 weeks old. During the following week, they could be spending increasingly longer periods of time outdoors, depending on outdoor temps. Week 4, they could be out all day long, again depending on temps. Week 5, they should be fine to move into coop full time, even in freezing temps.
Just FYI, my chicks are brooded outdoors with MHP heat source. By week 3, they are coming and going from their loft where MHP is, and playing in the lower level 8 x 12 run. They completely wean themselves off heat by the time they are 4 - 5 weeks old, with night time temps as low as 30*F.