I live in Northern NY. Farms keeping chickens year 'round here have their chicken coops built on to the farm house and share the indoor temps, like a mud room or an attached wood shed. If we get in a bitter cold spell, they can leave the door between the house and the chicken open to supplement heat.
You're right to be nervous. If you couldn't live in your chicken coop set through winter, you can expect your chickens can't either. Some colder climates still south of me get away with pilling the floor bedding extra thick to prevent frost bitten feet. But chickens live in the same temperature range as we do. If you want them comfortable, keep them about same temp as you have in your house, 65F to 70F. If you have long enough heating daylight and your winter days aren't mostly cloudy, you might be able to get away with chicken house that is heated with sun exposure. But if your nights are too long, like mine, it will get very cold at night.
A small window can let in sunlight for their circadian rhythms.
You could always install a small wood stove heater and fence it so they can't touch it for when your winters get worse than your hen house can protect them from. Be careful about heat sources. Chickens are very sensitive to toxic fumes and some can kill them quickly. Don't smoke a Teflon pan around them, for example.