I guess I totally fly in the face of what's considered "ideal" for chicks, and I don't relay this story to advise you what to do, but to reassure you that these little stinkers are tougher than we give them credit for. Last year, on April 1st, mine were put out into our un-insulated, unheated, unlit coop at 5.5 weeks old. I put a heat lamp out there for them because the temps were below 20. I kept jumping out of bed to run out and check on them.....all the long night through. They were fine - cuddled up in front of the pop door and not even near the heat lamp. The second night I only went out and checked on them once. Saw the same thing - chicks on one side of the coop sleeping peacefully, heat lamp on the other side eating power and looking for a way to start a coop fire. Third day the lamp came out - it snowed that night. We got our last snowfall on June 6th. The chicks were absolutely fine. They are now healthy, strong adults providing me with lots of eggs and entertainment. Didn't lose one, didn't stress one out, didn't have a sick one. The only creature stressed out raising that group of chicks was me.
Now, I'm not saying that you should unplug everything and evict them right now. You have do do what you are comfortable doing. But I do think that they are old enough to go without the lamp, especially if they are in the house. And since it won't be long before they will be going outdoors to live, this is a good time for them to experience the natural day/night cycles they will be living with once they go out to the coop. They need to learn to prepare to roost as it starts getting dark and learn to put themselves to bed and be settled once it does get dark. Cutting the lights while they are still warm enough is a good way to do that. With mine last year, I turned the light off about 5 days before I got ready to put them outside. They fussed and cheeped as the room got naturally darker, but then suddenly - for the first time since they'd arrived at our house - I didn't hear a single sound out of them for the entire night. No nighttime cheeps, no claws against the bare spots in the brooder...just blessed silence. And they woke in the morning as the sun came up, ready to eat and be active chicks all day long. There is more to heat lamps than the warmth to think about. There is also 24 hours of "daylight", not something they would ever experience with a broody hen in the coop.
I hope this reassures you a little bit. If you've gotten these chicks to 4 weeks with no problems, you've obviously been doing something right! You've got this, too!