Yes, you do. At least for the most part. When I first bought mine, I filled the trays, and just hoped for the best in humidity. I didn't have any way to watch it, either. And the chicks did pretty well in hatching despite me not ever knowing the humidity level.
But if you want,
Wal-Mart carries a thermometer/hygrometer so you can keep an eye on both the temp and humidity. It is digital, and runs on batteries, and it's only about $10. To see what it looks like, just click
HERE. If you can't find it in your store, you can certainly order it online through that link.
I bought one when the actual thermometer for the incubator was broken (thanks, nieces and nephews). I now keep that in the corner of my incubator right next to the motor for the egg turner. I can usually see it from above, but if not, a simple flashlight can show me what it reads.