If he's anything like mine, the girls are "grooming" him and ripping them out. The incoming pin feathers are pointing in that direction.
Watch him around the girls, especially when they are finished eating and just hanging out together and see if they are the culprits. If they are, he will never stop them and will always look like a goof ball.
You can also check him over carefully for lice or mites. They are pests you can treat for. The girls, not so much. Although you can try to coat the area in 'pick no more' and see if that discourages them.