i think i would start sending her a bill for your babysitting services. it might teach her to give her kid a key to let herself in or find somewhere to hide one on the property. or, better if they have a keyless entry to a garage or a door with one. that way you can't loose the key if you memorize a number.
if a child under the legal age is being left alone with no one given direct charge over them, then that is a problem i would take up with child services. in many states it can be considered neglect even if the child has been healthy and unharmed.
i would be very unhappy with any neighbor telling me to remove something that i can legally have and having to get rid of it because they don't like it. the day they pay my taxes, mortgage, and home owners insurance is the day they can make decisions for me.
we live in a country where those that think there should be a law against something are starting to get their way because too many of us have been backing down to the bullies!