What does your coop look like? A photo could help. My main interest is in wind protection down low and what ventilation you have. Is it elevated or at ground level? Is there a run with it? The size of the coop could help. How many chicks do you have? I'm trying to get a feel for what you are working with to offer more specific suggestions. Without that kind of knowledge all I can do is be genertic.
There are different ways to go about it. You are not trying to keep the area they are in warm, you are trying to allow them to keep themselves warm. You do that by having good ventilation to get rid of any moisture from their breath, poop, or the water they drink and keeping a wind from hitting them. What your coop looks like is important.
Most chicks are fully feathered at 4 to 5 weeks. At 6 weeks you could just move them to the coop and leave them, they will survive even with temperatures below freezing. So that is one option.
Many like to acclimate their birds. Your daytime temperatures are probably cooler than in your house but also above freezing. You can start taking them out now during the day to get them used to cooler temperatures. Many people are amazed at how well chicks even younger than 4 weeks can handle cooler temperatures. Take them out when you can stay and observe them. They will let you know when they get cold. Your best guide to how they are feeling is how they behave. I anticipate it won't take long for you to feel comfortable leaving them out there alone for long stretches of time.
When you do move them to the coop I'd leave them locked in the coop section only for at least a week. If you don't have a run to keep them locked in I'd go for two weeks. The idea is to try to teach them where home is so they might put themselves to bed in there when it starts to get dark. Some learn that, some you may need to train to put themselves to bed.
If you have electricity out there you can set up a warm spot and just move them to the coop now. As long as you can keep one spot warm enough in your coldest temperatures they will be fine. My brooder is in the coop. There are mornings I have ice in the far reaches of that brooder but I keep one area toasty. The chicks stay where they are comfortable.
For what it's worth, I've had chicks go through nights in the mid 20's Fahrenheit (-3 or -4 C) with no suplemental heat at 5-1/2 weeks old. They were acclimated, they had great wind protection down low where they were, and a lot of ventilation up high.