How hot is your heat lamp? You might switch to a 100 watt red bulb, especially at night when they aren't moving around so much and can cuddle for warmth under it. As they get real feathers, and the temperature outside gets more springlike, you might just keep it on at night. If they are in the shop, make sure they have bedding underneath them to keep out the cold a concrete floor can suck the heat right out of a body. Soon, it will be time for them to experience the great outdoors, and can maybe run back under the 100 watt bulb to warm up for naptime. (Ahhhh, naptime. It feels good at any age!)
More chicks will keep each other warm as well, if just a couple, then I wouldn't rush to eliminate the lamp. I think the recommendation is usually starting at 95 degrees, and then minus 5 degrees each week until it equalizes to the average temperature (in the coop or room, or outdoors), or their full feathers are all in, maybe 6-8 weeks.
I just moved my 2 week old chicks from inside brooder to outside mini-coop (not opened to outside yet, they need more feathers and we need sunnier days). I will miss their little peeps all day long, but can visit them to make sure the heat lamp is still working (and check on them at night too, it would be terrible if the lamp went out and I woke up to dead peeps.) I have a backup heat lamp bulb, just not enough space in the mini-coop to put two heat lamps.