It depends, how dirty, and with what? If they're caked with poo, I'd wash them. If it's just dirt and mud, or a few smears or specks of anything, I wouldn't worry about it. Guinea egg shells are much thicker and harder than chicken eggs, less prone to invasion by bacteria.
Guineas on the loose typically hatch clutches of anywhere from 15 to 30. They nest on the ground, in dirt, and nobody washes those eggs.
One of my guinea hens just appeared a few days ago with 17 keets in tow. Another is brooding out in a brush pile. Those eggs were no doubt, at least somewhat dirty, if not downright grimy.