She is not laying eggs so she does not need oyster shell. The baby chicks sure do not need any opportunity to eat oyster shell. I strongly recommend to not give her any oyster shell.
She was weakened a bit by her time being broody. The chick starter is higher in protein than layer or her regular food. It will help her regain her strength. Chick starter is fine for the broody hen.
The chicks can and will eat anything the mother eats unless it is just too big. And they can eat surprisingly big things. If a grown hen can swallow a mouse (and they can) think what a baby chick can actually swallow. As long as they have grit or are on the ground where they can get their own grit, you can feed treats. I would not feed the mama treats unless the chicks have grit, but if they have grit, a few treats will be fine.
Congratulations on the successful hatch. Exciting isn't it.