My guess is he was bored. That's the sign of a bored bunny that doesn't have to opportunity to express their natural behaviors. Lemme guess, you have wire floors and bare wire cages? Rabbit don't "play" the way dogs or even cats do. Usually they get great joy out of just doing the things that come naturally to them. That includes digging, running, chewing tearing and throwing objects, hiding in quiet dark spaces and searching for food. Frankly, a bare wire cage with a wire floor and a feed dish provides none of that. A rabbit starts to go a little crazy imo under those circumstances.
Luckily this is easy to fix. Toys are easy and free to produce, which is good because rabbits destroy them as part of playing with them. An empty toilet paper roll with an alfalfa cube in it, a paper towel roll with a carrot inside and the ends folded over, a piece of old carpet tubing stuffed with hay and dried dandelions, a plain brown cardboard box turned on it's side as a rabbit-edible shelter, a piece of natural twine with some apple, rose, pear, black/raspberry, mulberry or other rabbit safe wood branches tied into it and hung from the top of the cage... Personally, I have solid floored cages bedded with hay. This lets my rabbits dig and hunt for their favorite parts of the hay to eat every day, or arrange the bedding in their cage to suit their preferences but this is a personal husbandry thing and takes extra cleaning. You can also buy toys. I have had a lot of luck with lava rocks, cage kebobs and nut knobblers. No particular endorsement on brands or stores, just some quick links I found.
https://www.chewy.com/kaytee-lava-l...Y2QbYIqa1klrq6HsnUjMDMM43VYIOsirNKRoCLgbw_wcB
https://www.amazon.com/Kaytee-Perfe...9_5?ie=UTF8&psc=1&refRID=37VQQPJT8E81NHTVXJ4Y
https://www.walmart.com/ip/Super-Pe...30463&wl11=online&wl12=20585600&wl13=&veh=sem
https://www.amazon.com/Kaytee-Small-Animal-Knot-Nibbler/dp/B0002DJEYI
My rabbits have also absolutely loved these and these;
https://www.amazon.com/Kaytee-Count...9_1?ie=UTF8&psc=1&refRID=37VQQPJT8E81NHTVXJ4Y
http://www.lambertvetsupply.com/Sna...l90uVjBJ9yglmxnCIUh6w9thH7Sn2MGjPsxoCVqTw_wcB
But they can get a little gross after a while in cages with bedding or un litter trained rabbits (they get peed on) but they might do better on wire floors.
Another DIY or buyable option is something like this;
https://www.walmart.com/ip/Ware-Hay...54378&wl11=online&wl12=32801419&wl13=&veh=sem
Which will last as long as the metal doesn't rust through. I don't have any of these myself but I like them.
Rabbits have a strong instinct to chew, dig, tear and throw objects in their cage an envirionment. It's good for them and reduces stress. It also helps maintain healthy teeth and I personally find that the lower stress of having stuff to do leads to healthier, more manageable rabbits.
If you DO wanna put a litter box in, you will need to give him something to do other than chew on and throw the litter box. Then make the litter box too difficult to move. Otherwise he'll just keep taking out his stress and needs on the litter box because it's literally the only thing he has available.