They don't hate you. They're just scared of you. What would your reaction be if something 40 times your size chased you with a large fearsome object? Be patient around them. Walk slowly, talk quietly to them and get them to associate you with food not brooms.
