Why do my baby chicks hate me?!

Baby chicks do act terrified of people. That is normal until they get to be tamer with time. Some breeds really don't get tame very well.

But it sounds like you might have some aggressive chicks in there, which has happened to me before. So what I recommend is (and this holds true with adults- hens who are aggressive) the following:

If a chick attacks your hand, grab it. Pick it up and place it gently on the floor. Hold it down with your hands so that it must lie on its tummy (do this EXTREMELY carefully with chicks so you don't injure them).

Hold it there for just a few seconds and say nothing. Then, let it go.

You have just told it that you are above it in the pecking order. No chick or adult chicken allows another chick or chicken to peck it without doing something in return unless it is lower on the pecking order.

I once had a chick only 2-3 weeks old viciously peck my hand every time I tried to reach out to hold it. It was a Delaware. Needless to say I don't have them anymore. I have heard others say that the hatchery Delawares can be like that (not breeder Delawares).

I have "cured" aggressive hens of pecking me with this treatment I mentioned above. It is very effective and I can go right up to them and pick them up, carry them around like babies and dote on them afterwards. I am the top hen! You need to be the top hen too!
 

I noticed that they seem to have an instinctive fear of things that approach them from overhead or loom overhead, as a predator would (think hawk). It can help to find a way to approach them from underneath, such as putting your hand down in a corner of the brooder without chicks, then scooting it toward them near the floor to scoop one up from underneath. Try not to loom. Or if you have a cage with a door, elevate it so you are at a level more similar to them. Using a verbal call when you give treats can help them associate you with goodness. Some are just more the wild type. Good luck.