Zach,
That may very well work, then again it may not.
As you know, late hatch chicks are often smaller: much smaller. However, they still retain the same genetic makeup as larger earlier hatched chicks. Often you can breed them and the result will be normal size birds.
That said, anything is worth a try. Just keep using the smallest hens and you'll get there quickly. As long as you are using small hens it really doesn't matter that much how large the cocks are because the chicks are generally going to take on the size of the mother.... there are exceptions, but this is generally so.