You'd need to skin the foam insulation to protect it. Were it me, I'd make the nest boxes using foam-sandwich construction--works for superstructures on boats around here, fishing offshore year-round.
After carefully drawing out your plan on graph paper, you just make the nest box the size it needs to be, using, say, 1/4-inch smooth exterior-glue plywood (I don't like Aspenite or Advantech sheathing for this: too heavy, and no rigidity at thin dimensions). If you're going to paint the inside of the box, do that now, saving a final coat for after its assembled. Test-fit the assembly to make sure that everything works per your drawings. Make two sets of everything, for an inner and outer layer. Note that, depending on how you draw it out, the outer layer of plywood will be slightly larger than the inner layer. Leave some slack in the outer layer if you're unsure of your measurement; you can trim it off later.
Next, outline the box parts with furring strips the same thickness as your insulation (one inch of Tuff-R or blue or pink Styro is plenty) and about 1.5 inches wide. Glue and screw these down on the _outside_ of the box. Also, run one furring strip across the center of the bottom, back, and lid. Cut and tightly fit the insulation into these cavities, gluing down the Styro with a product called Styrobond, or if you're using Tuff-R, any kind of construction adhesive in a tube will work.
Fit the box together, and glue and screw it together. When the glue completely sets, smear the furring strips and insulation with more glue, then screw down the outer layers of plywood. Now you can paint the interior, and finish off the exterior after screwing it into the coop wall and caulking all the seams. Be especially careful to paint any exposed plywood edges, as these will wick in moisture and ultimately delaminate the ply.
I'm sure there are simpler ways of building an insulated next box (including just building a simply board or plywood box and covering it with spray foam insulation covered with a thick smear of aluminum mobile-home roof cement to prevent photodegradation and chicken-pecking). But with foam-sandwich construction, the bond between inner and outer skin and inner insulation forms a continuous I-beam, for great strength with minimal weight (and effective insulation). I've used it for many projects around the house (and for much of the house, for that matter), and there are scallop draggers and lobsterboats up and down the Maine coast with topsides built this way.
