Like others said, it's good to follow your gut if you have a bad feeling- but, if you want some reassurance I'll share something positive with you: My yard only has a 5 foot fence, and backs up onto a very busy public park. My coop is 8 feet tall. People look over all of the time and ask me what's in the cage, or want to show their kids the chickens. When I've been in the yard people have asked me if I sell eggs, chickens or chicks before. My back gate to the park is not locked, and my coop does not have a lock. We leave the chickens loose in the yard every day. I have never had someone steal a chicken, harm a chicken, or steal an egg. I think people are just curious. And some of them would rather buy a local chicken or fresh eggs, or want advice about where to get there own. If you have a coop with more than a couple of birds, maybe they were truly just curious about if you sell production birds for eggs or meat.
Although, if you have an icky feeling, I'd still follow my gut. I put a big 'in case of emergency save our pets' sticker on my front window. I listed both of my dogs- and also a BIG german shepard who sometimes stays here. Honestly, this just makes me feel better. If someone has their heart set on stealing, they will find a way to do it. The only thing you can do is prevent easy opportunities. I had my entire apartment ransacked when I was in college and completely cleaned out- because my landlord hadn't given us window coverings yet and all of our stuff was in plain sight. The door was deadbolted and neighbors were home. They just plowed right through the font door and splintered it to shreds.
I still believe most people are good. I think they would be likely to steal a chicken from a store if they were hungry, and if they were really interested in chicken keeping they would research it and get chicks.