You can also buy little solar lights for less than $30, they have a tiny panel that goes on the sunny side of your coop and is wired through a wall or window to the bulb panel.
We used to haul hot water to the barn before we got a generator, and you could leave a 5 gal jug of hot water in the coop to assist in warming the air mass if the power goes out during a severe temperature drop. We've had that a few times, once for 4 days!
Some folks have a wood stove with an attached hot water heater for such emergencies and I've seen people heat bricks on wood stoves to use as foot warmers, and they might have coop/barn applications. Whether you heat your coop water or not, it helps to have it elevated on conder blocks and a patio block, because the ambiant heat in the concrete holds for a long time.
The big problem during a power outage is fresh water, so it's important to stock what you might need for up to 3 days. When we had more than 1 horse, we kept 200 gal on hand at all times, and used it up in the garden over the summer, replacing to keep potable.
It's a real concern, I sleep better now that we have a generator. I used to fret about the animals every time the weather turned snarky.