Exactly.If your rooster grows up to be aggressive towards the kids, you have far worse problems than his spurs. If he's aggressive at all towards humans, simply get rid of him. You won't want to keep him. An aggressive rooster could still do damage with beak and claws to a toddler. Add to that the fact that people who are attacked by roosters when young often are really afraid of or dislike chickens for the rest of their lives--this is a chance you do not want to take.
As far as spur cap removal, you really only do it if he's accidentally hurting hens or can't walk well. And I say accidentally, because if he's hurting hens on purpose, he's gotta go. Our big rooster accidentally cut a hen pretty badly while mating, which is why we took the spur cap off with pliers. When it healed, it was too dull to cut anyone and about an inch shorter.

I had a rooster accidentally tear a big hole in my favourite hen's side. It was inches wide and very disturbing in looks.
Damage to the hen can often happen when they are molting especially.