Well egg size has no effect on fertility, large eggs are fertile too but if an egg is a double yolk it is extremely rare for the 2 chicks that form inside to actually live, they cannot generally hatch without assistance and knowing when or how to assist them is a tricky situation, I just hatch average sized eggs, you don't want to use overly small eggs either, just normal sized eggs for your hens.
As I asked in my previous post, do you know for sure they were not fertile? Just because the eggs don't hatch doesn't make them infertile, early quitters can look like clears when candling, you really would have to break them open afterwards and see what the insides look like to see if they were infertile or not, it would be highly unlikely to have 16 of 20 eggs be infertile if you have 2 or 3 roosters and see them breeding. All of the eggs could have been fertile and simply died during incubation due to an issue with the incubator or even bacteria getting into the egg.