Yes, the lamp needs to be on 24/7 to maintain temps. appx. as follows: 95 week one, 90 week 2, 85 week 3, etc., reducing by about 5 degrees per week. These temps. are average, as some do fine a little cooler. The chicks will let you know if they're too cold by clumping up under the lamp rather than scattered about all over.
If your brooder is outside and you have hot temps, within 3 weeks they would probably be okay for daytime (although they'd need the lamp as soon as temps fall for the evening). Most use the red bulbs because they are less harsh during the night.