Chickens take cold weather much better than they take hot weather. The cool spring and fall weather is their favorite, here.
Does having the windows open cause any draft on the chickens when they're roosting? Do they have draft free areas to hang out in on the floor when the windows are open? Sometimes keeping chickens out of drafts in a small coop is more challenging than in a larger coop.
If drafts aren't a problem, then you mainly need to keep them open long enough that sudden warm days don't over warm the coop. However, you can leave them open indefinitely.
This is very important, though. Do you have any other ventilation openings cut into the coop? Chickens need ventilation in the winter, too. It releases the extra moisture that can build up in the coop. That keeps their lungs healthy and keeps them from getting frost bite. Plus, they need a certain amount of fresh air for oxygen and to release excess carbon dioxide. If you haven't cut any other ventilation openings in the coop, don't close any windows until you do.