Beekissed told me a regular lightbulb would be ok for chicks kept in the house. Here is the info from Beekissed:
I've used both the red heat lamp bulbs and regular bulbs and they have the same results, but the plain bulb is easier to regulate on heat than is the heat lamp...not to mention a safer item to work with. With only 4 chicks, a simple light bulb will be all they will need...that way you won't have to buy special equipment. The chicks will sleep just fine with the light on and it won't keep them awake or make them agitated enough to peck one another. Chicks have been brooded for many a long time with just a plain light bulb and inside the house is the perfect place to use them. Now, if your brooder was outside and very large, the red heat lamp type of brooder light would be more needful and useful to keeping them warm.
I've brooded chicks outside in an open air coop in March with 30 degree weather with a heat lamp in a very large brooder but I'd never use one inside the house, no matter how chilly it got there. The ambient temps in a house usually don't dip below 50-60* and the brooder holds the heat of the light/lamp in very well. You can drape a towel over part of the brooder to keep the heat in but be careful of getting them too hot and without good ventilation.