I have had to shut it sometimes but due to weather not predator protection. One of my cheats though is that the two chickens in question often can't sleep in there due to my little cockerel having insane, ginormous wattles that he can't keep warm enough in the cold. So in my case it's even been mostly a daytime-only experience for the chickens so far anyway. When I adapt it for broody duty later on, I'll probably either do some more modding or just keep the door open 24/7 like you said (I have a predator skirt on it anyway).Do you shut them in the house at night or let them access the run 24/7? Thats what I do and it works really well