Hi, hope you are enjoying BYC!
If it's too small, it passes straight through the digestive tract. If too large may actually reduce digestive capability.
Here are a couple studies..
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7666813/
https://cdnsciencepub.com/doi/pdf/10.4141/cjas60-010?download=true
I usually let my birds get their grit from the ground once outdoors.
This last year is the first time (10+ years and hundreds of birds) I bought grit for (Marek's) vaccinated chicks and their quarantine period. All other years. bringing a clump of dirt (grass. bugs, and all) into the brooder to allow some resistance to build to whatever's on my pasture before heading out full time while also providing a free and natural source of grit, was my preferred choice. Since deciding that quarantine is a joke and not possible at my house with pasture grass coming in daily for guinea pigs, dogs traipsing in and out, cats too, etc.. I still have the chick grit unused, on hand.
I probably wouldn't switch back, but give more time for adjustment, perhaps raise the dish to reduce waste. Some billing can be caused by boredom.
They're such creatures of habit, my birds hated (feed) pellets when I tried to switched from crumbles. You did well mixing sizes and allowing for transition time.
My personal thought.. I bet they got no issues woofing down meal worms, etc whole at their age.. There are still smaller size and lager size grits in the package.. they are probably taking what they need and will ingest the larger particles of the "grower" size grit down the road.
Sorry for a semi vague answer.
How many birds have ya got? Do they have ground access or are they constantly secured in a coop? Anyone laying or getting close?
Hopefully you will get more feed back from some other seasoned keepers.