what can I do? How much can I trim without hurting him?
Sometimes in animal husbandry you have to do things that appears to hurt them but causes no real harm. The idea is to make life better for the animal with minimal discomfort.
Those spurs are interfering with his quality of life if they are interfering with him walking. I don't know how much you need to take off to enable him to walk OK. They will keep growing so you may need to take off more than the minimum.
Those spurs have a quick in them. Different chickens are different, I don't know how close to the end his quick is. Usually if I take off a quarter of the length I don't get blood, sometimes I can go further, I've never had spurs get that long and I don't know how the quick has grown in his.
The way I cut them off is to use a Dremel tool with one of those discs you use to cut metal. Some people start grinding from the end and stop when they get blood or get far enough. Some people use pet nail clippers. I wrap them in a large towel or sheet. I wrap around the wings but don't cover the head. You can if you want to as long as it is loose enough around the head so he can breathe. Then I hold the leg with one hand while cutting with the other. Occasionally I cut too deep and get some blood, usually just a few drops. I think that cutting blade is so fast it gets hot and cauterizes the wound so there isn't much bleeding. It's still a good idea to have a handful of flour or cornstarch handy to toss on and stop the bleeding if it is too bad. It may look a lot worse than it is.
Usually when I do hit the quick (which isn't that often) The rooster doesn't even flinch. I only had one that even acted like he felt it. It's not like you are torturing them. Your imagination is probably a lot worse than his reality.
After I finish I just turn him loose with the rest of the flock. He mates a hen to tell them "Girls, I'm back" and the flock is back to normal. It's not a traumatic event for him. It is quickly over and he can walk again.
My flock free ranges and we have foxes Fischer and coy- wolves... so I don’t want him defenceless.
Spurs of any kind are not really going to stop a hungry fox, Fischer, coyote, wolf, decent sized dog, coy-wolf, or coy-dog. If he tries he's just the first one taken. By cutting off the ends of his spurs he's no more defenseless against them than he was. He'll just be able to move faster and be more sure-footed if he is not tripping over his spurs. You will have helped him.