Most chick starters already have chick grit as part of the mix. If you're using a commercially bought chick starter, like DuMor or Purina, you don't need chick grit until the chicks are about 8 weeks old. If you're going to let them forage around, though, some chick grit wouldn't be bad. Just make sure it's for chicks (tinier pieces) than for full-sized birds!
I put a cake pan full of course construction sand in the brooder. They love it even if they do not "need" it yet. They appear to thouroughly enjoy scratching around and even dust bathing in it. Less than $3 for a 50lb bag.