I don’t know what your coop looks like, if you have electricity to it, or if you have grown chickens. I brood my chicks in the coop from Day 1. I have a built-in brooder so they are protected from the older chickens. It’s 3’ x 6’ and has good draft protection. I only provide heat in one area and let the rest cool off as it will. The far parts of that brooder can get pretty cool but they play in it anyway, just going back to the heat when they need to warm up. When it’s down around freezing I cover it better to reduce ventilation and provide another smaller heat source to keep the food and water area warmer for the first couple of weeks. You need to watch them and be flexible depending on what they do. At 4 weeks your chicks are well beyond this phase.
If your coop has good draft protection and you can safely put a heat source out there, move them out there. Just put the heat in one area and let the rest cool off. Of course be careful of fire but you have to do that in your brooder anyway.
I don’t know your conditions, including what temperatures you are talking about. It seems each batch of mine I do a bit differently depending in the time of year. During our triple digit heat wave last summer I turned the daytime heat off in that outside brooder at 2 days and the overnight heat off at 5 days. Our lows at night were probably in the 80’s. Even at night they were getting as far from the heat as they could.
In colder weather I often keep the heat on until 5 weeks, day and night. I’ve put 5 week old chicks out in an unheated grow-out coop with good draft protection and good ventilation when the overnight lows were in the mid 40’s. Before they were 6 weeks old the overnight lows hit the mid 20’s. They were fine.
I’ve had some between these extremes. I try to watch how they react to the heat to see what they are telling me to do.
I’ve seen a broody take her two week old chicks to the roosts when the overnight lows were in the mid 70’s. She could not cover all of them on the roost. The chicks she did not cover were fine, but they did have others to snuggle against, at least on one side.
I don’t know what temperatures you are talking about. This time of year my overnight lows can vary by 30 degrees within a couple of days. It’s hard to give specific recommendations not knowing your weather, what your coop looks like, or what other options you might have. A garage or some other building may give you options like 3forfree mentioned. Or maybe your weather is already warm enough even at night that you are past the worry stage.
With yours being used to being together I’d treat them all the same, as it they were 4 weeks. You don’t want to separate them since you have then integrated successfully. If you can’t acclimate them I agree you should be a bit more cautious on when you remove the heat, but they are pretty tough. Most of them are going to be fully feathered at 5 weeks.
With your younger ones at 4 weeks, if they are still in your house it is highly unlikely they need heat at all. Remove the heat and watch how they react. I think you’ll find they are fine right now.
Good luck! I know it is nerve-wracking, especially your first time, but they really are pretty tough.