... What I am noticing tho is when the temp goes down some the humidity goes up and when the humidity goes down the temp goes up...
That's basic atmospheric science. Warm air can hold more water vapor, so given the same amount of available water vapor, relative humidity will go down as temps go up, and vice versa.
It sounds like your setup is struggling to provide enough moisture to overcome the loss of humid air through the oven's vents. Is there a way to block them?