What type of bedding are you using in your brooder box? If it's pine shavings, my girls do the same thing occasionally, too. I think they do it out of curiosity more than anything else...silly little girls!
Yeah, they do that. They'll get over it, once they figure out its not food. Same with wood chips and whatever else you put in their space.
Just wait until you see what full grown chickens will try and eat.
Maybe someone else will have an idea. I just make sure their feeder is always full and wait for them to get over it.