Hatching eggs under my hen - Help

Hi, I'm new at this site and new to baby chicks. I've got 2 pet bantam hens that are both sitting on some eggs - just got a rooster a couple of months ago. I think its about day 20 and I just found a cracked, rotten egg under one hen. It was fertilized, but not developed. I removed it and cleaned up a bit, but I didnt know if I should move the hens and eggs or just leave it. they are nesting in hay and I'm rather afraid of the bacteria possiblity from the broken egg.