If you experience lows around -25F regularly throughout the winter, yes, insulation would help.
If you do not, don't insulate.
You need to think about what a bird needs not a mammal. Birds need lots of fresh, dry air. They can easily keep themselves warm if they are in a dry, draft free environment.
Speaking of drafts, a draft to a bird is not the same as a draft to a human. While you may feel a draft through that crack between the window and it's frame, the bird won't. If their feathers are not ruffling open in the breeze, they aren't feeling the draft and what you have is another good source of ventilation bringing in fresh air. You need A LOT of ventilation in a coop.