Definitely worry about the ants! Ants have attacked my mealies many times. They killed and ate many when I didn't discover the infestation in time. I don't know the best way, because what kills the ants can kill the worms too. I generally try to destroy the ants outside of the box and the ant trail. As they get disturbed, I spray a towel with Trader Joe's cleaning spray, which I find to be an excellent ant killer, and wipe down the sides of the box repeatly. As ants climb on top of paper and veggies in the mealie box, I toss them out too. Getting rid of the ants is a PITA. Afterwards, I put DE around the box.