It’s personal preference. You do need to be able to access all areas, maybe to retrieve eggs or injured or dead animals. I don’t know that you have to necessarily crawl, I’ve used a rake to reach some things. There is a huge convenience in being able to walk in there; that cannot be denied. For the animals themselves a couple of feet tall is high enough. For your convenience, higher is better.
There are a couple of reasons to not make it too tall. Those things can get pretty heavy and hard to move. The higher they are the more materials you need so can they get heavy. Another possible issue is that the taller they are, the more wind area you have and the higher the center of gravity. In a strong wind they are more likely to blow over if they are not pretty stable. You might consider a hoop house design and not use any solid materials up high that will catch the wind to keep wind area and weight down.