If you feel that your chhicks will be too cold, go ahead and put a heat lamp inside the coup. Make sure that it is far out of reach of hens so that they can't knock it down. Chickens are hhardier than you might think. It also depends on what chhickens you have. RIR are cold hardy, but if they don't have all their feathers they could get a little you could also try inulating your coup. Good luck!
I'm in MN and as long as you have cold hardy breeds pullets have no problem. Use a red heat lamp at night for the next week or two if it makes you feel better but remove it ASAP. Fried chicken is good- except when you don't want it! I only use LEDs on timers in my coop due to low heat on them. In reality, heat is a bigger issue than cold- I won't let my chickens out when it's below zero- mainly due to frost bite.