As long as they are all raised together, I don't think you'll have a problem. I would worry a little bit if one of your EEs is a cockerel, if you plan to keep them in the same coop. I have a LF EE cockerel (Bandit) in the same coop with Poof, and it's a little hard on her when he tries to mate with her. (My husband is building me a new coop for her, but the project has been set back a bit.) She is well accepted in my flock because she is the first chicken I owned. She was the original flock that came in with 3 other silkies. When it was time to add, the pullets and cockerels were about the same size as the silkies, since the silkies were about a month older. I sold the other three silkies because they were all roos, and Silkies were not a breed I wanted to procreate. Poof is my only Bantam, and she does well to provide comfort to my young pullets and cockerels as they come in the coop. I set a bench in my coop that has a low bar that sits maybe 6" off the floor. At night she will either hop up on the bench seat or she will roost on the low bar.
I'm keeping her because she will eventually become broody and be my natural incubator, plus she's just fluffy and cute. I will probably add another silkie pullet at some point since I have a local source, but not for a little while.
As for adding a cockerel later, I wouldn't if you have any other roos. A friend brought me a silkie cockerel because she was over-run with boys. My boys knew exactly what the silkie rooster was, and attacked it. I took the cockerel out immediately. Couldn't keep him.