I had the opposite happen. When I first got mine, they were picking out the small small pieces of the crumbles and leaving the larger pieces so I used a blender to reduce the sizes. At about one week of age, they started picking out the larger pieces and leaving the powder, so I quit grinding it. When I switched from crumbles to pellets at 19 weeks, I tried mixing the pellets with the crumbles, thinking to transition. They started picking out the pellets and leaving the crumbles. I had to let them run out of pellets before they would finish off the crumbles. They are all different and I sure cannot explain it.
You might try reducing the size of the pellets some, maybe with a blender, and gradually increase the size. Mine still spill a lot of pellets on the ground, but unlike the crumbles, they will clean them up. I let the pellets run out about once a week in the feeder so they will clean them up, plus they clean out the food in the feeder so it does not get too old and moldy.
I do think the pellets are more efficient so it is worth some effort to get them to switch over. I also know chickens do not always listen to reason or do what you want them to.
Good luck!