Pics would help people tell you whether your predicament is what you think it is. 
Also, give this a read: http://wiki.answers.com/Q/How_do_you_stop_a_rooster_from_attacking
It's a really good summary of how we handle our rooster. So far, it has been working and we have never had to watch our backs or defend ourselves in earnest (and our roo comes from a background of kids teasing/kicking him). He's actually been getting much better...not spooky with us or starts getting his hackles up like he wants to start something.
Basically, watch his behavior AND yours. You need to be careful not to present yourself as a threat OR a rival, so try seeing things from his perspective when you approach, handle, and are otherwise in the vicinity of the flock. If he ever starts to get a little miffed with you (you'll know - watch his body language and those feathers), pick him up. Pet him, talk to him, walk around with him, and when you put him down (make sure he's not struggling when you release him) give him treats. Regularly approach him with treats so he associates you with good things. While you can handle and touch the hens around him, AND SHOULD, don't push them to the ground patting their backs (he'll think you're mating them, which is a direct challenge in rooster speak).
The hardest part will be training yourself and your children. For example, don't walk up to him bent at the waist only and stuff a handful of food in his face. My husband took several hard, blood blister-inducing pecks to his hand when he did this. Then he did it again, and I saw him do it, and then he turned to me like "why's he hate me???". I said because he was lording over him and shoving his hand in his face. Once he learned to stop a few feet away, squat down, offer food, and call to them. Once he started doing that, no more issues.
So just think about how you look to them when you're doing you're own thing. Another item to keep in mind is that children are loud, erratic and fast....and that quickly flusters the heck out of a roo. So make sure you teach your children how they need to behave around a roo...similar to if they were being introduced to a strange dog; calm, quiet, and with slow and deliberate movements.