That's true, some people are mean for fun, and some do things out of desperation. Alot of fake homeless too, (college kids taking handouts to pay for liquor) . I agree with getting a camera and locks. Get a no trespassing sign too, like others have said.
Do you have any homeless people in your area? A neighbor of ours had similar experience and put up a game camera and caught the guy. It turns out guy was homeless and desperate for a meal.