There are very few deranged vicious roosters that attack humans for no reason. It isn't in the interests of their long term health and tbh, one has to admire their bravery if they do.
Even a large rooster isn't likely to weight more than ten percent of the human he attacks. He doesn't have a height advantage or a reach advantage.
From what I've read on this forum in particular the most common reasons for aggression from roosters are, feeding his hens and picking up his hens.
The important point here is, no matter what you may think, the rooster thinks the hens are his.
It's a very important point. For some keepers the inconvenience of coming to terms with the fact that the rooster considers the hens his and expects you to act accordingly is enough to make them decide that there can only be one boss and that has to be the human. I can understand this but in such cases perhaps it's better not to keep roosters.
The next common reason for rooster aggression is that while roosters are juveniles they can be the most affectionate and endearing creatures and their human keeps respond to this treating them more like pets, picking them up, cuddling them and generally making a fuss of them. Unfortunately roosters grow up and their flock status changes along with their behavior. Some keepers cannot adapt to this change.
Imo and in my experience the most sensible advice has been given by cetrarchid and DobieLover and that is wear suitable clothing at all time, much as one would when say handling Mink. A mink is far more aggressive and will happily bite an end of your finger off given the oppertunity. It's your responsibility as a keeper to make sure you dont and nobody else who comes into contact with them gets injured by the animals you keep.
This article may help you understand why your rooster is aggressive towards you. It won't give you a solution to your problem directly though because there are so many variations in your and the roosters behaviour. I have roosters that are absolutely fine if I do A, but not at all impressed if I do B.
It's a long article and there is a lot of information in it. It's up to you to decide which pieces of information are relevant to your keeping arrangements.
https://www.backyardchickens.com/articles/understanding-your-rooster.75056/