Once they have all their feathers in then you will need to wean them. They're going to be used to the lamp and once you turn it off they're going to huddle again and be cold. I usually wean them a little at a time. It usually takes about a week or sometimes more depending upon the birds...
That's still pretty cold for the little chicks. You need to keep the lamp going and put them in the coop if they won't go. They'll be fine once all their feathers are fully in and it starts to warm up more.