I'm sure someone else will post some good ideas. I wouldn't have had any myself except that the power went out the second week for quite a few hours, and we were worried about them.
I got a gallon jug and filled it with very-warm water and placed that in the brooder. I replaced it after a few hours, and it seemed to work great. We also used a double-bagged pouch of water at the same time, which they seemed to like better and snuggled up closer to, but at the age they are now (almost 3 weeks) they would probably destroy that.
Can you not use electricity at all? Or just avoiding heat lamp? I asked because we also use a heating pad with ours as well. Oh, and I have seen them dig little holes for themselves in the litter and cover themselves up with it too. Not sure if that was for warmth, but I have seen them do it.
Just wanted to offer!
trish