I would go ahead and make the door big enough that if by chance you go on to get the big guys you are covered. I wouldn't sweat the insulation loss too much. As long as your coop is draft free when door is closed you should be fine. Another draw back to a small door is on hot summer evening when that is a cool place, sometimes a dominant bird will decide to roost there, not allowing others to pass by and get in the coop.
