I lose power all the time during winters (did just a couple days ago) usually for at least an hour, two hours, sometimes over 10 hours and honestly? I just keep them in a warm area. We have a woodstoves, so if possible I keep their brooder (simple cardboard box) nearby for warmth. If not that, then I cover the brooder with blankets, towels, whatever. The chick's body heat should build up in there. Haven't lost a chick yet to any of our power outages, except two one cold winter that were outside. Had a heatlamp in the coop, power went out in the middle of the night, it was 20 degrees out, and two died from suffocation/trampling.
Inside the house though, the only thing I worry about is my incubator(s).