Depending on what your chicks are doing, they may not need it.
Mine are going on 5 weeks, and they've been off the heat lamp since 3 weeks old. But, I live in GA, they are in a non-drafty room, and they were hanging out in the opposite end from the heat lamp, staying as far away from it as possible.
I let their behavior be my guide, they obviously weren't feeling chilled. They've been happy and fine without the heat lamp. They definitely don't need a night light, chicks raised under a broody hen don't have one. I do keep a light on during the day, and weather permitting they've been getting some outside time too.
One thing though, when you first take your chicks out to the coop, if they seem reluctant to roost in there, a little light bulb can encourage them to go in. I used one with my first chicks last year til they got the hang of "coop is safe place to sleep" instead of "coop is big and scary! DO NOT WANT!"