Would you hear the chicks if they started peeping loudly because of being cold?
If yes, then feel free to leave the lamp off as long as they are comfortable and happy.
Chicks that are cold (or otherwise distressed) make a very loud, annoying peeping sound. I think it would wake up most sleeping...