Dealing with a frightened flighty rooster is a challenge. If you are abrupt and use force, even throwing a towel over him, it can escalate the fear he feels. The objective is to calm his fear before handling.
Understand that a good rooster shows respect by avoiding you. He does this because he sees you as far superior and dominant. Countering this sets up such a cognitive dissonance that some roosters have a dramatic melt down. Psychology is everything when dealing with roosters.
Approach him as you would a frightened child. Before reaching in, talking softly and reassure him. Then extend your hand toward him, talking softly. He will withdraw. Keep talking softly and extend the other hand, but do not touch him.
When you notice he is starting to relax, slowly extend one hand to touch him on the breast. Wait until he accepts that. Then reach the other hand in and touch his wing. If he is calm, extend your other hand to touch his other wing, then gradually get a firm hold of him and walk him toward you out of his crate. Making "shhing" sounds calms most chickens. Do this if he begins to get agitated.
Do what you need to do, avoiding any unnecessary handling, especially cuddling, and then release him back to his crate.