Check out a cinderblock heater like this one https://www.backyardchickens.com/t/428896/my-diy-water-heater
You can make it for less than the $15 you spent on hand warmers and as long as you can sneak a heavy duty outdoor extension cord to the coop, your landlord will never know that you use...