I think a huge part of it is just the resources needed to do a good job breeding the birds. Few of us have the facilities for hundreds of birds, which is really needed to do a good job with this hobby. Me personally, I think it would be a lot of fun to be involved in serious breeding, but right now I'm limited to under a dozen birds, and realistically, I may never be able to keep more than two or three dozen.
You can do well enough if the dozen birds is a dozen breeding hens. 3 cocks and a dozen breeding hens is a workable number.
Then if the birds are already in good shape, we can hatch 32 in the spring and fall. Work through the growers fast etc. Some do fine with some strains hatching 40 chicks in a season. These are in strains that are already in good shape. Heck you can get rid of 30% of them at 8 wks, and another at 12 wks. It is not as if all 40 are grown all of the way. There are ways to work it out. 20 in the spring, and 20 in the fall. 10 breeding hens, a cock and a cockerel, and a partner . . . .
It does help to have a partner or two.
I hatched out 120 in a single breed this spring. That is no small amount, but it was an appropriate amount. I will hatch more from them this fall (God willing). It is a project though. Projects require numbers. Single or simple colors in already good shape does not require nearly as many.
Bantams are better options for many. There is nothing wrong with bantams. Some are productive layers, and some proportionally are quite meaty. They grow faster, and eat far less. A much more practical option for many. There is no actual logical reason that excludes them from consideration. I would have been just as content, working with a flock of bantams. I often consider them for on the side.
What it would costs to feed 12 large hens, you could feed 24-36 bantam hens. The housing requires less space and investment.
And you are right, it is not for all. It may not be for me at some point. Time will tell. My remarks are usually based on that some say this or that, but that is not really where there interests are.