I only shut their pop door on nights when it is below 25. They love to be on their own schedule, which does not always fit mine.
Their coop is inside a run that is like Fort Knox - surrounded by railroad ties, 5 layers of buried wire, locks, bolts, solid roof, etc. Plus the run is behind our solid 8ft fence with motion lights everwhere and a house full (and neighorhood) of cats and dogs. I will never say never but I feel pretty secure with our set up. However, if something ever does get into my yard and then run, it will get into my coop so leaving the door open is not going to prevent anything.