My husband's turned out to be a rooster, but he was in love with "his" bird...The thing is, he's very sweet and docile when he's just hanging out in the house with us. He turns into a completely different bird outside when he has ladies to impress (and my husband isn't there).
Since your husband is the one most attached to this rooster, and he doesn't have trouble (at least so far), I do hope you have talked with your husband about the matter. Maybe he could be responsible for all outdoor things that involve the rooster. Having the rooster inside a pen, and only your husband enters that pen, would be a way to keep everyone else safe.
Maybe keep the whole flock of chickens in a pen you don't go into, or maybe make a pen for just the rooster and have it next to the hens' area so he can interact with them through the fence but cannot actually get into the space where you and the hens go. Then your husband could do anything that needs to happen inside the rooster's pen, and you would not be attacked.
It is usually best for chickens to have time outside, and to be part of a flock instead of living alone. But in this case, having you considered having the rooster live inside all the time? Since it sounds like he is not a problem there.
Unfortunately that guy can draw blood even through my jeans. I've been told you can have their spurs filed down, but here in Los Angeles it's almost impossible to find a vet who will even see a chicken.
That is definitely a problem. Filing the spurs should be a job that you & your husband can do, without needing a vet. The usual advice is to wrap the rooster in a towel, with his head sticking out one end (so he can breathe) and one or both feet sticking out the other end (so you can work on them.) It is probably a two-person job, with one to hold the rooster and the other to work on the spurs.
You should be able to blunt the ends of the spurs with a dremel tool, or maybe even with a hand-held file. You don't want to go too deep, just like you don't want to cut a dog's or chicken's toenails too short, but you might be able to just cut off the tip, the way you would with a dog's toenails, to save time with the dremel or file. The spur is bigger than a toenail, so a small saw might be the right tool to cut off the pointy end. (I have never done it personally, but I've seen quite a few articles about it, and there are also youtube videos as another poster pointed out.)