Your eyes adapt, through experience, to seeing the difference very early on. At 6 weeks, most single combed breeds already give themselves away. The little cockerels are the first to sprout reddish combs and little red wattles under their beaks. The pullets don't generally show much redness until much closer to their time of laying.
Males often have stouter legs and look taller, with head held high. Their role, by nature, is to serve as lookout and so both their bodies and their behaviors reflect this nature assigned role. The taller, more upright stance is also required as it is they who will mount the females in breeding.
The female, by nature, is often lower to the ground. In many breeds, her coloration is far less intense. Better to camouflage her in laying and brooding. Her lower stance also makes laying an egg, closer to the ground, much easier to accomplish.
After many years of keeping chickens, your eye becomes trained to note these differences and make the distinction.