Yes, the hens will lay fertile eggs for 2 weeks after a successful mating. Here's a nice guide to show how to check the eggs for fertility:
https://www.backyardchickens.com/t/16008/how-to-tell-a-fertile-vs-infertile-egg-pictures
And do mark the eggs! My MIL didn't and one day she collected...
You have 2 options then. Give her some eggs to hatch or break her broodiness by placing her in a wire bottom cage for a few days, with only food and water. No nesting materials! The average in the cage is 3 days, before the hen gives up. I usually feel sorry for the hens or start thinking about...