The issue you need to think about is not grit but parasites. Earthworms are carriers of intestinal worms that chickens can get.
Granted, they will eat worms when foraging. That does not mean you really want to add to the load, or introduce it this early.
Mealworms from the bait store cost...