Nah, they will be fine. They’ve spent all winter acclimating so even though 3 degrees is pretty cold, it’s probably only another 10-15 degrees colder than a normal night(if your days are around 20 degrees).
You don’t really even need an insulated coop. Just as long as there are no drafts. Airflow is important, but windchill is a real thing. In these colder months, I block up the vents that are in line with the roost. But the ventilation is still excellent.
Our coop walls are just plywood, no insulation, and our chickens have been fine, like yours. They also do a good job of snuggling with each other to stay warm at night. Some of my girls even bury their head under sisters wings to warm up combs.
Before a cold night like this, try to give them a little carb snack just before bed... corn, scratch, oatmeal, barley, etc... their bodies work a bit harder to digest the whole grains over night, and it actually increases their body temp. Good little hack, and they’ll love the treat!