Because there's virtually no regulation in the amount of silica it can have before it's labeled "food grade". It has to be under a certain amount but there's no regulation on how far below. The more silica, the more dangerous it is to breathe. Silicosis scares the crap out of me, and it can take 15-20 years to show up, and you can get it from even an extremely small dose of silica in the lungs.
If it's on the floor, and I wear a particle mask when I clean, I'm probably going to get most of it. A normal 'dust mask' won't cut it because the particles are too fine. Unless I wear a particle mask EVERY TIME I go into the coop, I'm at risk. My creatures dust bathe. I can't be sure they didn't do it right before I walked in, or that they won't while I was in t here.
I've watched a relative die from silicosis. It was brutal and ugly, and that's the main reason I won't go near it. It's why I don't use clay or sand in the catbox, either.
The other reason is because the higher the silica concentration, the sharper the edges, and I don't want microscopic razor blades anywhere near my soft tissues, like my eyes, nose, mouth, lungs or innards. I've seen stories of people eating it. I haven't seen reports of the long term health of those people over a 15-20 year span from the time DE was added to the diet.
I don't have that many birds. Pest control isn't time consuming for me. I'd rather do it the very much old fashioned way and keep the scary stuff as far away from me as possible.
Please note, I'm not judging anyone for using it or not using it - These are simply MY reasons for going nowhere near it.
-Spooky