No, chickens don't peck on paint ....unless you've got some big sloppy paint drips that dry, and then they might try pecking at those. Just wait until the paint's dry and there's no more paint smell before putting the box into the coop with your chickens. The real hazard for chickens (like all birds) is inhaling the fumes from wet paint in an enclosed area.
I use plastic nestboxes, mostly, but I now have one wooden nestbox that I recycled from an old drawer that I could no longer use as a drawer. I painted it, inside and out, with latex semi gloss paint. It will be easier to clean, and the paint also seals the wood somewhat to give mites fewer places to hide.
Before I put in bedding, I always put a light layer of poultry dust or DE down on the bottom of the box to deter mites.