I don't trust digital thermometers. What you want are some good, dial meat type thermometers, and you want to calibrate them in ice water and boiling water. Or at least some of the little "mercury" type that come stapled to a flat piece of plastic with the incubators when they're new.
You also want to place the incubator somewhere that the temperature doesn't swing, an empty closet would be good.
Be very careful wrapping with towels so that you don't block either the bottom vents or the top ones.
To change the temperature on the kind of unit you have, you want to turn the knob 1/4 turn at a time, and then check the temperature a few hours later (make sure you tighten the wing nut back down so that the arm can't keep moving). Ideally, you would have set it up dry before you wanted to incubate, and then watched and adjusted until it was where you want it, but that's neither here nor there now. Also, little bits of styrofoam or pieces of egg shell can get lodged in the vents of the bottom half, you want to make sure all those vents are clear.
This model (I'm assuming you have a 1602 if you have a turning knob instead of digital) is my preference specifically because it is SO stable with the temperature. I think you have a number of factors going on here, there's no way that it should be swinging that wide temperature wise.
I hope that's helpful.