I have the exact same problem. My rooster isn't too terrible with the hens though, but he attacked me the other day.
Some people say you can break a rooster of this behavior, others say it's not possible.
As for myself, if my rooster attacks me again, I am going to attack him right back and...