I would pull it out, she can't be comfortable with a screw dangling from her body.
Trim all the feathers down to the skin so you can see what your dealing with, make a little incision and work it out. Chickens have healed from much more dramatic wounds than that.