When our family use to raise burly tobacco and hang it in the barn to cure, the metal roof and rafters were full of red and yellow wasp nest.I know plenty of people that just go nuts when they see a wasp and get stung. I was told when I was only around 5 to be still when one gets close to you. I did, and to this day largely ignore them unless they land on me. When they do, I just stay still and they will fly off w/out stinging me. It creeps people out if they see this but it works.
As long as you didn't bump into them, then they would just hang on their nest and watch you.
My grandfather use to use the larva in their nest to trout fish with on the South Fork of the Holston River.