I can't see anything whatsoever wrong with caulking (etc) and insulating, as long as you are not panelling over your vents. Air leaks don't make good cold-weather ventilation anyhow (you get drafts and condensation). And insulation is always a plus.
BTW it may not be worth the bother of...