He is a lot tougher than you give him credit for. You are not going to injure him by giving him a side-foot soccer kick. You aren't drawing blood, you aren't breaking bones. At most you are giving him some minor ( and I mean MINOR) bruising which should remind his small brain that you are the boss. Mostly, you are damaging his ego and he needs it damaged

Personally, I catch mine ( gloves and sleeves help) and carry them under my arm and with my free hand give their necks a slight twist with just enough pressure that they know they are in trouble. It's the double-confinement that makes them think twice. When I just *hold* them, they still fight and attempt to get away. I've even had them try to bite my free hand ( hence the neck grab). I think of it as the equivalent of confining a criminal and jerking his hands into handcuffs behind the back. You wouldn't be gentle, but you wouldn't be mean either. You just get the job done with the amount of force it requires.
Remember the brain is small.
Also, the more often he wins, the more persistent he will be in trying to win. If he has intimidated you a few times already, it will encourage him to fight you even harder. Keep that in mind when you decide to settle this issue. You MUST win, regardless of what it takes.
The last time my rooster drew blood, I was on antibiotics for 10 days because it became infected even after cleaning. Do you want to chance that with yourself or your family?