These pictures aren't clear enoiugh for me to tell what sort of wasp.
It does look like a newly started nest, very few wasps. You can probably just knock it down and they'll go somewhere else.
Painting the undersides of boards with glossy paint will discourage them some. Supposedly so will...