I am by no means and expert on turkeys, as this is our first year with them, but I have had no problems with my turkeys. Mine have all lived with chicks at first, with the last of the turkeys still in with chicks. The others are all out free ranging it. I had absolutely NO problems with the poults dying.
Ours have all been healthy and the youngest are telling me it's time to graduate to outdoors life.
Mine have been eating non-medicated food, as I understand they should NEVER be fed medicated food. They get the same chick feed that the chicks eat. I am not sure what the problem is with the medicated food, but they shouldn't have it. This is the only think that I can think of that could be a problem for you. Did you feed the poults medicated food? If so, stop feeding to them.
We started out with six bronze, then five royal palms and a slate, then three each of slate and red bourbon. No problems with any of them with food or living with chickens or geese outside.
If medicated food isn't the problem, maybe it is a problem with the eggs the hatchery is using. If you still would like to try turkeys next year, try getting them from a different source that use eggs from a different supplier and see how it goes.
Since we have had no problems with our birds I don't have any other thoughts for you, except to not give up. They have been a joy to raise from just hatched to large birds with definite personalities and good entertainment...and soon to be dinners....at least some of them.