Got Thermal Mass?
Thermal mass is a heat sink. It equalizes to the temperature around it. But based on how much mass there is, this equalizing (change of temperature) can happen much slower than the surrounding environment.
Lots of different stuff can function as thermal mass: rocks, bricks, stones, water, glass, sand, concrete. Anything that can absorb heat. In an incubator jars of water or rocks/bricks are the most common.
In my incubator I used 4 large paver stones (concrete bricks). These added a lot of thermal mass... I know cause they are heavy! What this also did was make it take forever to heat up my incubator to 100 degrees... since it had to heat up all that mass too. Air does not hold heat well and therefore can heat up and cool down very quickly. If my box didn't have the pavers, it would have heated up pretty quickly. But conversely, it would have cooled down equally as fast... especially when the lid would be opened.

As an experiment, I decided to measure the temperature in my incubator as it cooled down after I unplugged it... to simulate a total power failure... no light, no fan. Inside my incubator I used a glass bulb thermometer exposed to the air and a second glass bulb thermometer inserted into a plastic egg filled with liquid dish soap (a homemade egg-o-meter). I set up my webcam and time elapse filmed it all night as it cooled. Only a light outside the incubator was used to shine light through the observation window so I could read the thermometers on the webcam. So this isn't exactly scientific-labratory accurate, but the results are clear.
(pic captured from webcam at end of experiment)

Here are the results:
elapsed time temperature (air / egg)

start 99/100
15mins 91/99
30mins 88/97
45mins 86/96
1hrs 86/96
1.5hrs 82/95
2hrs 80/94
2.5hrs 79/93
3hrs 77/92
3.5hrs 76/91
4hrs 75/90
4.5hrs 74/89
5hrs 72/88
6hrs 70/87
7hrs 70/86
As you can see, with all that thermal mass in the box, it held temperature surprising well. I'm not sure if an incubating egg could withstand cooling to 86 degrees after 7 hours and then be brought back to hatch... but I think one could definitely survive a lot longer in my incubator verses one without any thermal mass in it.
So if you have any room in your box, it's a good idea to add more thermal mass.

Cheers!