I'm just leaving the brooder plate on with my hot-weather chicks. It can't over-heat the brooder the way a lamp can -- they just go in and out as they need it. :)
What sort of bulb are you using?
I've gone as low as stealing the 40w bulb from my bedside lamp when I needed to take just a little chill off a 70+ night when some of the 2-3 week chicks didn't have any feathers.