It's up to you, of course, and you don't give any indication where you are located, but even here in Northern Wyoming I don't use any supplemental heat. They simply don't need it....not even here where we experience extended periods of sub-zero temps, 60mph+ winds, and snow. They have the kind of down and feather coat we pay a small fortune for, and they know how to use it! As long as your coop has lots of ventilation for moist air to get out, and they are dry, they'll be absolutely fine. I leave a couple of windows in their coop open all winter....the windows I have are on all sides except the north, and I close the one where the weather is blowing from. I have operable high vents that are also left open. It's all about ventilation more than heat. Warm, moist air rises. If it rises and hits the cold air above them it condenses and drops down onto the chickens, a sure recipe for frost bite. I also have roosts that are 2x4s with the 4 inch flat side up so they can sit on their feet and protect them.
Too many coop fires are caused every year by heat lamps for me to even want to think about it. I did use one the first time I put my chicks outside, April 1st of last year, but they ignored it for two days so it came out. And our last snowfall wasn't until June 6th that year, so that should tell you something about how adaptable they are to cold. Your coop is small enough that they should be quite comfortable with the body heat they put off, and at 4 feet high there I don't see a way for a heat lamp to be used safely. Dust and dander will accumulate on it, and at that height it would be very easy for them to fly into it.