yes, they need the heat all night long. I don't usually start weaning them off the light until they are about to go outside and have fully feathered out.
The idea that red light is not visible is a myth. We can see it, reptiles can see it, and chicks can see it. Just look at their pupils and you will see them reacting to light. It is more soothing, but if they could not see it then my chicks wouldn't be eating at night...and I've seen them easily maneuvering the brooder with just a red lamp.