Yes it is a heat lamp. We've had a few nights in the 20's and I keep my flock cozy.
I'd really reconsider the heat lamp. Unless you're raising new chicks in a brooder in the coop, they definitely don't need it and it's more of a fire hazard than anything else. They can fall, the bulbs can shatter, someone recently even had a bird somehow get lodged into one. Chickens can tolerate below freezing temps without a problem as long as they have protection from wind and rain, which your coop looks like it handles without issue.
On the plus side you really do have coop space plenty for quite a good number of birds (most of us have nothing near as big, so we're jealous!) so even without adding a second coop like you're suggesting you should be able to grow your flock without the issues that many others face.
I'd adjust the roost too to be higher with rungs a little more separated but if you're satisfied with their current sleeping habits then that's not a huge deal to me.