Does that mean the Bantam Dwarfism gene is recessive?That would probably work.
You may not need the Jersey Giant, depending on what size birds you are able to get from your other crosses.
There are several genes that control size in chickens, and different breeds may have a different combination of those genes. I read of someone once that crossed two bantam breeds and got standard-sized chickens!
I would try to pick the largest birds among your other crosses, and raise some Jersey Giants, and look to see whether the Giants are actually bigger than your crosses before making a final decision on whether to use them.
I got what looks to be a Dwarf out of breeding two LF Silkies together.