The crazy thing is I have 3 other roosters that are the same age and have been with me since they were chicks and they don't try to attack me. They and the head hen try to stop him. I haven't hit him - just try to walk him away from me. He got after me today when I had the hose in my hand - he got sprayed and man he hated that.
Even when of the same flock (assumed similar genetic background and rearing conditions) you will have variation. On genetics side many genes involved even within a family and even identical twins can behave differently do to slightly different experiences growing up.
The walking away part I suggest you refrain from. Getting into his space OK but giving him the feeling of being pursued is not.