Welcome to team Quail!
I live off grid, and personally I'd say your most dependable light would be one that is hooked up to a real solar setup (Even a very small one) with a separate deep cycle battery to recharge/power the light(s).
Of course, it is the more expensive option, (even when done cheap) but I find it much more reliable than those classic garden solar light style lights with the very dim LEDs. They just don't function up to snuff in the colder months in my opinion. (it gets pretty cold here though, so you may have more luck)
Even the most basic solar kit would usually come with auxiliary lights to hang that connect straight to the charge controller. You would just have to get a deep cycle battery to hold that charge, and then release it over night. There are a few kits around 40-60$ mark i was looking at that include a small battery bank, 2 or 3 lights, and a small panel to keep it charged. Those come with batteries, but i haven't used them in the cold, so i cant speak to that yet. I will say that I was considering them soon for my coop soon as a lighting option. Something like those or a more upscale solar setup with lights and a deep cycle just seems like it could have a more broad spectrum use for the cost to me.
Interested in knowing what you choose as a solution.
Good luck!