Partly it depends where the window is. If it's somewhere that'll pour a cold breeze right at the chickens when they're roosting, I'd close it down sooner than if it's, say, at the opposite end of the coop. (Remember you need open ventilation pretty much ALL THE TIME, even in winter, just not necessarily as much as summer -- if you have no vents other than the windows, don't close them all the way except in brief circumstances)
Try getting a cheap max-min thermometer to put in the coop near the roost (like, at roost height). It may well not get NEARLY as cool in there as it does outdoors. As long as the coop air is dry rather than damp (which is largely a matter of having sufficient ventilation), they are completely fine down to freezing, at LEAST (generally colder too).
Good luck and have fun,
Pat