Most definitely yes, they can - if the door closes before dusk! If you have it on a timer, be sure and adjust it throughout the year such that it won't close until after dusk/dark. I believe some people have a light sensor that triggers the door - this may alleviate the risk of locked-out chickens!
Search the forum for 'automatic pop-door opener' or some such - there are several informative threads on the subject here - good luck!
This is the main reason I won't have one ever. I simply do not trust them and would not want to take the chance! Plus one of my girls always stays out - even after dark - to make sure I'm coming to lock up their coop. I think they like me to tuck them in
My girls got closed out once since I've had it. We'd gone out to dinner and there they were on their little chicken porch all gathered outside the closed door...lol. Thankfully my run is pretty daggone secure. But I always check on them every evening anyhow to do a head count, so even if one DID get shut out, I'd discover it within an hour or two. Mine is on a timer, so as Coloradomike said, you have to be sure and adjust the time as the days lengthen. Just today I changed mine to open at 6:50 am and close at 7:15 PM.
DH made mine from a drapery motor (can't remember the name of the company off the top of my head...but you can just google drapery motors). We bought our timer at Lowes. That was it. We just used the same door we'd been using before on our pulley system pop door. If you want more info, feel free to PM me. I do have a pic. of our set-up on my BYC page.
Only time my girls ever got locked out was at the change of seasons when you have to adjust the timer. But if you pay attention to these changes you will be there to make the needed adjustments with no problems.
I just sold my automatic opener. BUT the only reason I did so was because my current set up is totally contained. The only thing that could get into my coop and runs is a mouse or snake but they wouldn't last long with my girls!! The new coops are open to the runs so that the girls can come and go as they please.
If I still needed to open and close the door for them I would have the opener from Foy's that I used for 2 years. It was such a good feeling to know that we could go away for a day or 2 and didn't have to try to find someone to let the girls out or lock them up at night. It was well worth the money I payed for it!
I must say, an auto door would be nice. But. I really enjoy the excitement of the flock when I open their door in the morning and I wouldn't want to miss counting heads at night before I close the door. I seriously can't sleep until I know they are all in and settled. But that's just me.