Mine pecked the smaller ones at first to establish dominance but they all live together peacefully now. Make sure your banties are old enough that the pecking won't make them bleed. If they bleed the whole group will gang up and kill them.
I agree with Bantyhugger. Watch when you mix them to see that none are seriously injured.
It's funny, but once they are grown, size of the chicken isn't always that important in determining pecking order. It is not unusual for the larger chicken to be lower on the pecking order. Depends more on the spirit of the chicken.
Editted to add: Separate feeders and waterers when you mix them seems to help keep down the conflict.
I don't understand all the problems mentioned here, I raise bantams, and do it for the roosters. It's fun watching them mature and see what beautiful color patterns they get. I've not had all the problems mentioned here. Just know how much I love my roos.