I think you answered you own question. If you think it's too cold out, it's too cold to move them to the coop without the heatlamp. How many are there? That will effect how cold the are in the coop too.
You can put up a mini perch if you think they are too small to be roosting. Mine are...