I have some 3 month old babies that are still fairly small. It dropped to the single digits so I brought some of the banties inside for the night. But now that they are in I worry if it will be harder on their systems to come in and then have to go back outside. Any advice?