The only horses I've seen go lame after starting them late, aside from accident and injury, had conformation flaws which made it inevitable or someone decided to start doing a whole lot of tight circles endlessly until it wore on the joints.  Circles at any age, including on the ground lunging, is going to make a horse more likely to go lame and amongst some groups of riders endless circles are their favorite training method.  I went to showing western gaming horses to avoid that crowd.  Most barrel horses are not started until they are 3, walk/trot the first year while introducing them to the correct type of turns, cantered and shown lightly with no serious goal of winning for their 4th year, and then maybe they will be running serious patterns by their 5th but more likely the 6th.  I can count on one hand the number of barrel horses started on that time table that went lame without it being a fence, trailer, or injury from another horse.  Most continue to run in to their teens and I've been on a few 20-25year old barrel horses that will still run a pattern.  You can't say that about most racehorses and they aren't dealing with the sharp turns and stops of many western disciplines or the jarring a jumper would experience.  I've also seen plenty of quarters, thoroughbreds, arabs, and crosses of those that weren't worked with until they were 4-5 and it took no more time than it would have otherwise.  The difference is you expect them to make more progress quicker since they are older so it seems to take more effort and time when it's probably less than you'd spend waiting and making slow steps with a younger horse.