That does sound reasonable.
Sometimes we see people who want to add a heat lamp because the temperature is 50 degrees (yes, above zero, in Fahrenheit) and they are sure the chickens must be cold :lau
So I have learned to always ask for actual temperatures!
You should definitely measure the actual temperature inside that coop before you add any heat this winter. I'm guessing they will not need any extra heat.
If the chickens do not have heat in normal times, then they will also be fine during a power outage, and you will not have to worry about that.
If you bring them inside for very many days, they will start to get used to the warmer temperatures in the house, and they will have more trouble when you put them back in the coop again.
Have you measured the temperature inside the chicken area? It is probably several degrees warmer than...