Put some ventilation holes along the top of the wall on the far side from where they roost (cover them with hardware cloth to keep out varmints). That will allow relatively warmer and moister air to vent. If it seems too windy, you can put a deflector on the inside or outside to reduce drafts...