I will not get into whether you should or not. That is purely your choice. What is right for my circumstances is not what is right for someone else. I do wish you luck with it.
I would use warm water, somewhere close to incubator temperature, and soak it loose. A little at a time if I had to. Be careful not to tear the skin. After it was loose, I would put it in a cup of warm water and pick off bits of membrane or whatever is there, but do not worry about gunk left on. Keep the head above the water. You don't want to drown it. Then leave it in the incubator until it dries off. You don't want it getting chilled.
The reason I would want to get the membrane off is that I would not want to leave something that the others would peck at.
I once shrink-wrapped a chick by opening the incubator too early. I soaked it in a cup of warm water, picked the membrane off, and let it dry off in the incubator. Whe it dried the fuzz was matted down with that gunk. I put it with a broody and other chicks the same age. Within a week that gunk was worn off.
Good luck. And congratulations on the hatch.
