Dust them with Sevin 7% garden dust, you can find it at TSC or any big store like a Walmart in the garden section. It's a red bottle. You need to dust the birds with it, clean out their coop and dust that, and then repeat the process again in seven days since the Sevin only kills the adult lice and dusting again kills all the ones that have hatched since the first treatment.
Lice can damage feathers and they make the birds itchy and uncomfortable, so it's definitely worth treating.
This is the lice treatment.