Yes!
They need a roost to sleep on.
It could be a thick tree branch, or some 2x4 planks of wood, and it should be a few feet off the ground, and higher than the nest boxes.
You should allow about 10-12 inches of roost space per bird.
You don't want them sleeping in the nest boxes at...