I do not have electricity for my coop or run...
Your chickens will do fine.

If you are really worried (and it sounds like you are!) bring them warm oatmeal in the morning and feed them a bit of whole corn an hour or so before you lock them up but make sure the bulk of their food is a balanced feed.
Don't close your coop up so much that you trap moisture in there, that is what will cause them to get frostbite. Make sure the coop has ventilation. They are more likely to get sick with respiratory issues in a coop that is tightly closed.

I used to worry so much about the chickens in the winter but then i realized, they have been around for thousands of years and no one had electricity before the 1900's. Also, chickens aren't mammals so they have different ways to keep warm. It's easy to put yourself in their place and know how cold you would be, but try to look at it logically. I think electricity in a coop is MUCH more dangerous than the cold.
I wish i could give you a hug and convince you it will be fine!!