If they are eating enough hay I wouldn't be too worried about it. Hay is necessary for their teeth and is supposed to be available all the time anyway so I don't think them not eating their pellets is a cause for concern right away. However, if you start to notice they are not eating their hay then there may be a health issue coming to the surface.
I've been recommended by several vets to only give bunnies 1-2 oz of pellets per day anyway. They should have constant access to timothy hay or orchard grass hay (alfalfa is for young bunnies only). During the summer I also give them a large daily helping of fresh grass as well as fruit.