Ditto what has been said previously that the recommendations are likely based on research that was done for commercial flocks (or even small-farm-sized flocks) and has been modified as more micro-flock owners have gained experience with smaller spaces. And the importance of the combination of your coop/run design with your management style.
I've been keeping backyard micro flocks for about 7 years now (~3-10 hens/flock), and have helped create and manage a few dozen clients' flocks as well and had a similar experience to what Cheep N Peep said - I started with the previously-recommended 2 sq ft/bird indoor space and 4 sq ft/bird outdoor space and quickly realized that flocks in an urban/suburban setting needed more than that to stay healthy and happy.
What I usually tell clients and teach in my classes is some form of the following points:
1. You should plan your coop/run to give them enough space and protection from the elements such that they could be confined in it 24/7 for 365 days a year (technically required by many city ordinances) and still have plenty of dry, easy-to-keep-clean, predator- and pest-free space to spread out and do their chickeny thing.
2. In my experience, 2-3 sq ft/bird is enough in the coop (more is fine,
though with our New England winters you don't want to go TOO big so they can still build up warmth in the winter), provided you have at least 10 sq ft/bird in the run (more outdoor space is always better, if possible) AND...
3. that your run is covered with a solid roof so you don't just lose all that usable space every time it rains, and preferably tall enough for you to be able to get in there to shovel out snow in the winter.
4. Don't trust the bird numbers given by cheap coops you can buy online - they are based on numbers that could work fine for a typical small-farm flock of 30-50 birds, but they DON'T scale down for our backyard micro-flocks (those coops also are rarely predator- and pest-proof, and tend to be made from materials that break down within a year or two).
The reality is that you can get away with a slightly smaller space if you balance it with heightened dilegence in your maintenance, but only to a point. If they don't have room to move around and spread themselves out a bit, especially in winter, you're much more likely to create a few health and behavior headaches for yourself. I've found the plan-for-more-space to be a vastly preferable option.