Straw is great if it won't get wet, since it would then mold, and if it is sufficiently thick to stop the drafts - I think it has an R-value of 50 (that may be a whole bale thickness). How about a piece of rigid insulation board? Even a tarp secured with a staple gun - as long as water won't seep in and pool. I've used Great Stuff for small leaks. Pretty much anything that will block the wind. You might also use lots of straw bedding in the coop to help keep them warm - not hay, which doesn't have the structural airspace that traps heat - once the drafts are blocked. The key is good ventilation without drafts on the birds, and dry.
