If they are on their own, no hen to take care of them, I think they do, for a while. Put it in one corner of the box so they can get under it if they are cold, and get away from it if not. I like the kind that has a clip on thing, and a reflector. A "shop light" with a 60 watt bulb. Not an LED bulb. Those make lots of light, but not much heat.
But if they have a hen to take care of them maybe you don't need the light. Others here can advise you on that.
If you put them in a box, they are going to be pretty messy. Put some newspaper on the floor of the box, and I like to add some fine wood shavings to soak up the moisture from their poop. There may be something better than the wood chips, but that's what I've always used.