There are turkey whisperers out there that seem to know at about two weeks. I am not one of them. I can start to imagine differences in size, leg size, and snoods at about 4 weeks, feel a little better at 6 weeks. By ten weeks I think I have a pretty good idea, and by 12-14 weeks I feel pretty confident about almost all of them. By 20 weeks you should know for sure.
Toms are larger, have thicker legs, and more developed snood and caruncles (bumpy necks). Some people have talked about the shape of the head and the softer face of hens. I like to wait until I see the snood hanging down
I've gotten pretty good by judging the snood size when they hatch. The males will usually have a larger one than the females, although it's kind of tricky when they're still that little -- sometimes a male will be small compared to the rest of it's broodmates, so that can throw me off. For sure, I can tell by the time they're 6-8 weeks. Lagerdogger is right -- the males are usually bigger, more aggressive, and have larger snoods, where the females are more petite. To me, they just have a "girly" look to them.