While I totally understand not wanting a boatload of mice around, I hope that "kick a few" wasn't literal. We don't need to be cruel by maiming them.
I want to help you. You HAVE To make your food storage bins secure. I use metal bins with weights on top of the lids. Truly, this goes a long way to keeping mouse numbers low to nil. It is THE ONLY LONG TERM solution. All the death traps (whether literally a trap, poison or whatever) not only have downsides [unintended victims, grizzly deaths ...] but perhaps most importantly for folks on this site, only deal with a snapshot in time and your task will never be done.
In addition to ensuring no food around except when feeding your birds (take up food at night), you might also want to obtain MouseMagic by Bonide. It is a nontoxic mouse repellent made with spearmint and peppermint - I don't put it very close to my birds sleeping quarters as birds can be very sensitive to scents that are harmless to everybody else. Reportedly, cotton balls soaked in peppermint oil also repel mice. Again, keep a distance away from where your birds sleep.
To the extent you can, also prevent access to the building where your birds are. Hardware cloth...steel wool stuffed in any little cavities where somebody could get in....
Making an environment less appealing for a species causes that species to seek greener pastures elsewhere - I follow these practices and rarely have mice. And when I do, the numbers are so low it simply doesn't concern me. I do hav-a-hart the few I encounter and let them go. Or they just go on their own, once realizing it's not worth their time to hang out where my birds live.
JJ