Ground hogs don't usually move. They put down roots and after a few years there can be several acres of ground hog mounds and tunnels. Predators can employ ground hog burrows and tunnels to by pass your fence. The greatest danger however is skunks, foxes, coyotes, possums, mink, and even the occasional raccoon setting up house keeping in an unused ground hog nest. I have seen areas of pasture heavily used by ground hogs that become difficult to cross even in a jeep. The good news is ground hog dens is a good excuse for not cutting the grass.
