As long as your feeding good food, I doubt it'll affect egg size.
Ours all started out small, too. Don't worry - they'll increase in size.
I don't know how long it took for our BR's to get to their maximum egg size, but after hitting it, they laid very consistent-sized eggs, comparable to a medium/large in the grocery stores. Though each one layed slightly different colors (shades of brown) and some were speckled, while others were plain. It was really cool to see.
We've got 2 leghorns that also lay very consistently, though every once in a while, we'll get one the size of a quail egg, and then other times, we'll find one that looks like it was laid by a brontosaurus. (ouch)
We've got one Columbian, and she's a very regular layer, but her eggs are HUGE and cream-colored, but her shells are very thin and fragile, and not smooth - they usually have sort of a raised ridge around the middle.
Bottom line, you're about to see all kinds of wonky eggs......that's all part of the fun.
Good luck
Brian