No, hens generate heat and have very effective insulation, eggs have neither. Unless you meant too cold for the eggs rather than too cold for the chickens. Still no, it isn't too cold for eggs until it is cold enough that they freeze. That is somewhere much less than 32F (0C) if you gather your eggs at least every late afternoon or evening because it takes a while for them to cool. Especially if hens set on them periodically as they are in and out of the nest to lay eggs. Freezing still isn't necessarily too cold. I had one frozen egg last year. I peeled the shell right away, enjoyed looking at it for a few min, let it thaw in a bowl, and fried it.
You can check for drafts with your fingers or face. Or sifting something fine (sand or flour, etc) down where you are checking, or a strip of something light and flexible that will flutter. All, except maybe the fingers, are pretty much overkill. If it the air movement is so slow that your fingers can't detect it pretty easily then it isn't a draft. Peace of mind is nice to have, of course, this kind of overkill doesn't hurt anything.
Making sure they are warm enough has much more to do with making sure they have all the feed they want. Especially, full crops at night. That is where the body heat comes - both fuel for generating heat and digestion itself producing heat. Edit to add, all day is even more important when the days are short because they can't eat in the dark.
And enough water because good blood volume and circulation does a lot to keep all parts of the body warm (like combs and feet too).
