Hello there and welcome to BYC!
Sounds like you are really enjoying your birds! They are so much fun to keep. Once you start a flock of chickens, you wonder why you didn't get them sooner!
BantamLover X2 Chickens can adapt to the coldest of temps if given the opportunity. Rarely if ever do they need heating in the coop. Chickens come with built in furnaces and all those feathers and can withstand brutally cold temps. There are many members here for Alaska that see Minus 40 on a regular basis and keep their birds in uninsulated coops, no heat, and even no electricity. And they do just fine.
The most important factor in keeping your birds warm in the winter is good ventilation. You want to keep your birds roosting low to the floor in quiet air and 1 square foot of vent space per bird in the eaves of your roof. While your chickens are sleeping, there is a lot of moisture building up from all the pooping and the breathing. This warm moist air needs to go somewhere. With good venting, it will rise up and catch this positive air flow and go out the roof. With out good venting, this moist air is going to fall back down on them as water or frost causing the birds to be wet, chill and get frost bite. Keep your bedding clean and dry, remove all water at roosting time so you are not adding to the moisture in the coop. Dry drier driest. The birds themselves omit heat. So as they all snuggle up together, there is a nice bubble of heat surrounding them. Just make sure to seal all cracks around the root bar to stop all drafts.
Never close off all venting even on the coldest of nights. If it is going to be a very windy night during the winter, you might close off a few of them to slow down the movement of air. But you still need this moist air to be whisked out of the coop.
Chickens need to get outside everyday in the winter. If your coop temp is too much warmer than the outside air, they will be stuck in the coop all winter. This will lead to all kinds of sickness and respiratory ailments.
If however, it is planning on getting down to 30 or 40 degrees lower than your AVERAGE over night low, then you can add a small heat lamp. So if your average over night low is 10 and it is planning on getting down to Minus 30, you might consider a heat lamp until the temps return back to normal over night lows. You are not trying to heat the coop, only add heat around the birds, bringing up the temp a few degrees. ALWAYS permenantly attach a heat lamp. Do not rely on the clamp as they can fall and cause a fire.
You can also tack an old towel to your roost bar in early winter. Chickens lose heat through the feet. So warm feet mean warmer birds.
So just let them adjust to your temps as fall turns to winter. Your birds will do just fine if you don't try to keep them indoors too much. Let them decide when they want to come back to the coop and they will appreciate it if you shovel paths in the snow so they can get out for some exercise on those snowy days.
Enjoy this new adventure you are on and welcome to our flock!