You're correct in thinking that the 6' x 4' will be a bit tight. It isn't just that they'll jam together in the cold, but in southern California they'll need a place to sleep at night away from each other for the heat.
I have the same size chicken house and I have six standards in it. Last winter it was great because they kept each other warmish by cuddling together. Right now the nighttime temperatures are staying in the high 70s (Not too bad. I know you folks from humid areas wish it would cool to that at night.) and they roost with a lot of spacing between them to stay cool. But they pant a bit before sleeping.
The key thing with the house is ventilation. In your climate, more is better and you might consider an open coop rather than one solid wall. If you do it with hardware cloth it's well protected from predators.
Mary