Don't let him be. When he comes toward you, walk at him until he turns away. If he doesn't back down or tries to actually attack, a quick (but gentle) boot away should stop him. Let him see you handling the hens. Don't let him come at you when you do. Don't let him mount a hen when you are there. You need to convince him you are the alpha rooster.
All that said, there are some that just won't give up. If you have a really mean one that you can't trust or that won't learn, he needs find a new home, either in another flock or in freezer camp. We have friends who had a rooster that just would not stop attacking their daughter. For some reason, he got it into his little rooster head that she was some sort of threat, even tho she never really did anything to deserve it. But he would not learn, no matter what they did. He isn't in their flock anymore.