Karen, sometimes what appears clear on candling at that stage can just be an embryo whose development was stopped right at the beginning, before any veins formed. It's almost impossible to tell if it was actually infertile, although sometimes after you determine it isn't going to grow, if you crack it open carefully (hit the egg with a knife, dont hit the egg on the side of the counter) so that the yolk stays together in a bowl, you can sometimes see just a hint of red around the blastoderm-meaning that it was fertile but did not develop.
On the other hand, in some cases, a rooster can lose fertility or just not like certain hens and the owner not be aware of it until many, many eggs just don't develop. That's why it's a good idea for the owner of the breeders to re-test fertility at home to be sure they're sending out a good product.