The ones in the ground are ground wasps there are also a hornet that looks like a yellow jacket that makes a paper nest like white faced hornets. There are also yellow jacket wasps that make the grey comb like nest as wellThey are not yellow jackets. Yellow jackets rarely nest above ground.
Hornets' nests are spherical and covered with a paper like substance.
You are dealing with hornets.
Yellow jackets normally have a hole in the ground leading to their nest.
Hornets have spherical nests attached to a structure.
I work in a honey bee store and my husband also takes care of wasps and hornets. Do it at night or early in the morning before dawn. Use a red light to see because they like honey bees are drawn to light and you will get stung. There are many ways to safely remove nests without getting stung 

