There is no one set answer to this. How well feathered are they? Will they be moving to a safe, secure, dry and draft free grow out house or pen with a coop? What are the current weather conditions for your area? Are you still providing supplemental heat to them? How many birds do you have and how large is your brooder (asking as if the brooder is too small/crowded that would be a big point towards moving them sooner than later)
*I* move my babies out at 5-6 weeks - at this point they have been fully weaned from supplemental heat as I do not use heat outside of my brooder in my current setup. I'd have moved this year's batch at 4 weeks, but didn't have the new setup ready to go yet. Next year my chicks will be started in an outdoor setup with a heat plate or the Mama Heat Pad (haven't decided which yet) because I was able to pick up a new unit that will be able to be located close to the house where electric is an option (my coop is on a part of the property with no electric).
You can start taking them out of the brooder for extended day outings. they don't have to stay in the brooder all the time. By four weeks, they may not even need heat during the day if the temps are in the 70s.
If there are no adult chickens in the coop to contend with, why not move them out there now? Just rig a heat source for night time for a while until the chicks feather out completely.
We've been taking them outside and letting them walk around in their coop for a couple hours every day and since it's been super hot around here I've taken off the heat lamp. They seem to be okay without. I was planning on moving them officially in about two weeks just to let them feather out a little bit more.