That's fairly normal - the lowest ranked bird is often the worst bully.
Yes intervening too much can be a problem, because sooner or later the chicks need to join the flock (unless you opt to house them as a completely separate flock), and repeatedly separating them just sets back the progress you make. Yes it's hard to watch hens chasing or attacking chicks but the chicks also need to learn how to deal with the bullying. A few ripped out feathers, or scared squawks, or being chased away from food for a short period of time... I consider all those acceptable. Actual injury or endless harassment (where the chick appropriately runs away or takes cover and the hen continues chasing, or guards the food to the point that the chicks can't eat) is not.