If they are day olds, then yes they need the heat lamp. 90 to 95 the first week, decreasing by 5 degrees a week.
Let your chicks be your guide. Huddling under the light, too cold. As far away from the light as they can possibly get, too hot.
I am new too, but yes I do think that you should leave them under the heat lamp all day. Place a thermometer in with them to know if they get too heated. Day old chicks should be kept at 95 degrees for the first week and then lower the temp by 5 every week thereafter. Also, watch for cool breezes and drafts since they will be outside. They will move elsewhere in the brooder if they get hot. Good luck!
I have a remote thermometer in my brooder. I place the sensor where the chick congregate. They are not laying under the light but beside it. The chicks prefer 92 degrees. They where born 3/8. Just an experiment. I have always followed the guidelines but thought I would try something different. John