To be technical on the subject, a hen is at least one year old vs a pullet which is up to one year old. Pullets are still developing and often have "issues" with their eggs that hens do not. For example, a couple of my pullets still occasionally lay double or even triple yokers, or eggs without a fully formed shell. Unless there is an actual developmental issued with the bird (genetic defect), these tendencies dissipate considerably once the pullet has aged sufficiently to be considered a hen. This can most certainly contribute to less fertility in pullets, and this is why I weigh all of the eggs I collect and consistently check the fertility of the eggs I crack for consumption.
And yes, young hens....meaning 1-2 years of age...are often deemed "more fertile", though I know of several poultry keepers who swear by their more mature 3-5 year old hens for reliable fertility. I suspect that part of the reason people believe that younger must = more fertile is because the volume of eggs laid typically diminishes as the bird grows older. The perception is that there is less fertility when in fact there are simply fewer fertile eggs to be harvested. But...I might be too presumptuous on that fact.
A non-genetic advantage of hens over pullets is that they are more readily receptive to advances from the cockerels or roosters. My pullets still run screaming when the boys try to mate with them, whereas my more mature girls know to hunker down and let nature take its course. I guess you could say that psychology has its place in the equation too...not just genetics.
So yes, genetics are important, and it's a subject I've studied enthusiastically ad nauseum, but there are other less scientific, behavioral factors to consider too. And sometimes those smaller chickens that have hatched out of pullet eggs do in fact catch up in size to their non-pullet-egg-hatched siblings. My largest Bielefelder cockerel came from a pullet egg. He was the runt of the litter by far but has displayed exceptional growth, weighing almost a full half pound more than the next largest cockerel. Not that's some fine nature & nurture at work, in my opinion.
Oh...and sorry for running on and on about this. I'm a total nerd.