If you feel that they cannot stay warm enough, you may consider keeping the litter inside and taking them out to the doe about twice a day at least until their eyes are open or a few days afterward. About that time they usually have enough fur and mobility to huddle together for warmth, possibly bringing them in on colder nights, if necessary.
This is what I do. I take the nest box and all in, and keep them in my hall closet. When I take the babies out to their mothers, most of my does jump in the box as soon as I put it in the cage with them. I then busy myself nearby until I see that the doe is done nursing and has left the box, at which time I take the box back in the house.