I have read things that indicate that this is a secret of gamefowl keepers of old, a trick for regulating size in a line. If a line got too heavy weight wise, they would select fall hatched pullets to breed to. For the reasons you mention, they don't get as big. The smaller the pullet, the smaller the egg, the smaller the egg, the smaller the chick, and the smaller the chick, the less head start he has. Don't know if this has actual genetic ramifications, don't see how it could, but sets up a feedback loop of sorts that could regulate size.
I regularly have orientals hatched in September. With some shelter and feed provided they seem to do better. Instead of a hen that turns on them and makes them fend for themselves at a young age they have a hen that broods them basically all winter, they don't game up until early spring, so I don't have to individually house a young bird that is still growing in the coldest months, which means they are able to conserve heat with brooding/huddling behavior in the coldest part of each 24 hour period. It does take more feed as there aren't bugs available.