Megan,
They shouldn't have muffs in the first place so breeding it out shouldn't be something you should have to work on. I think I would be finding more bantam hens. They could have been part of a project, nothing wrong with that unless you're selling the first generations of the project as purebred. That's when you're more likely to have those things pop out.
I guess you could breed all the hens this year and keep all the pullets. Next spring (or this fall if the faults are huge like muffs or they begin laying brown eggs etc), go over each pullet for faults including egg color and type then cull out the original hens and the pullets that don't make the cut and breed only those pullets without these faults back to the rooster. You may not have many hens but if you get even 2 or 3 good pullets, it will not be such a set back. We all have issues in this breed to work on. I've had my own problems to have to decide what to about it. Some faults are more serious and more difficult to overcome so you just need to determine whether to hold on to what you have (you know what you're working with now) or start again with a potential for more surprises.
The more chicks you hatch from your brown egg layers and the others with faults, the better chance for getting some chicks that are better than those hens but there is always a potential for more later on so you have to decide if you're willing to keep them all for a few generations or are you wanting to raise chicks to sell. I'm not entirely confident about the brown eggs, meaning I don't know how hard it will be to breed it out. If it's recessive, I think you would need to make sure to keep track of any chicks from those lines and never mate down the road. I'm thinking that the brown egg gene could be there for multiple generations just setting there waiting for a mate to also carry that gene and produce more brown egg layers putting you back to square one. Also, if you sell any that may carry this gene to other breeders, and their birds have always laid nothing but blue eggs but carry the gene for brown eggs, they may also have offspring that lay brown eggs and say things about you that you're saying about the breeder you got them from. It's a tough choice whether to continue to keep the gene in the breed or stop it in it's tracks so there's no further damages.
Stuff happens and I've had chicks that had obvious faults that I have culled that will never be used in anyones breeding program (either eaten or given to my friend who loves pretty chickens and roosters). I'm pretty sure that will never end but I also feel as long as I do the best I can to breed the good in and the bad out, I'll be slowly improving my flock and once I have a number of good (maybe not faultless) Araucana's, I'll stop buying from others and work on a line I can call my own. Lot of stuff to think about and I worry all the time too.
Cathy