Lots of great advice already, plus some ideas I hadn't thought of that we're gonna start using, like re-using those nylon/plastic feed bags as drop cloths.
We toss the waste - spent bedding (shavings, shredded paper), poop, straw, everything - into a compost pile. We currently have 3 piles going now, all basically just open bins made out of scrap wood wrapped in either old chicken wire or hardware cloth, that we keep a distance away from the house near the back of the property. Rats and mice haven't been a problem, yet. We do see the occasional mouse out in the tall grass or in the brush, but we have lots of hawks, rat snakes, and get the free roaming dog or cat from time to time. Haven't ever had one in the house.
Ants on the other hand...
I've heard about using mint, too. We have a couple of varieties of mint that we planted right in the ground and let spread wild around our peach trees. Not sure if it's helping keep anything away. Might repel rats and mice, but I know it doesn't work on ants since fire ants built a nest right under a chocolate mint plant.
We keep the chicken waste in a separate pile and plan on giving it about a year, til next season, before we use it in the garden.
As far as burning trash, our county is pretty relaxed about it, usually requiring a permit only for large burns and certain things. We aren't supposed to burn plastic or certain plants like poison ivy that can potentially harm people or livestock (lots of farms around here) that might be downwind. Our county actually does recyclables collection. We've done small burn piles of scrap wood that we can't re-use. Other than that, we keep our trash in bins outside away from the house til it's time to haul it to the dump, which is a couple miles away. Saves us a few bucks a month on having somebody pick it up. And re-use everything that we can.