Your best bet is to keep them in the coop during extremely bitter temperatures. The idea isn't so much to keep them warmer, but to keep them out of the wind and keep their feet off of the frozen earth.
If you can give them a draft-free but well ventilated area, and a roost that allows them to sit fully on their feet at night (a 2x4 with the wide part as the perching surface works well), most chicken breeds will do okay at those temperatures. You really want to reduce moisture in the coop at these temperatures, as it is one of the major factors in frostbite. Removing droppings and not heating water above 33-34° will help a lot. Good vents that won't create drafts are really the key since you will have some moisture just from their breath.
Be aware that moving them in and out of heated areas during very cold days is actually harder on their immune systems than just letting them get used to the cold. If you do move them to a heated area, it is best to take them back outside only when it warms up quite a bit. Dealing with large swings in temperature (due to taking them into heat then back out again) in a short period of time is stressful for them.
Hard-feathered breeds, silkies, birds going through a hard molt (with naked areas on the body), and frizzles have the hardest time keeping themselves warm and special accommodations may need to be made for them.
Beware confining them too tightly for too long. Boredom can really do a number on chickens and you'll probably have social problems/picking/etc if it is left too long. If you need to confine them consider adding 'enrichment' such as a head of cabbage or something else for them to focus their attention on.
