Probably...but we just did the NYE hatch-a-long and did a LOT of candling. It was an educational experience for our family, and we loved seeing the progress of the chicks developing day to day. I only candled ALL of them 3 times (days 10, 14, and 18)...and the days in between the others would take turns by incubator row so we could see daily growth. We had 19 of 29 hatch and most of them were shipped, so ours did fine with the handling.
Just make sure to wash your hands first! Also, they can tolerate the lower temps briefly...it is no different than mama hen getting off the nest to eat/drink/etc. Some incubators even have a daily cool-down mode. It is the temp spikes that cause more problems than temps going lower. Also, once you put them in lockdown, stop candling because that is when humidity is most important.