Snap traps work better if you wrap some string around the bait pan and really work the peanut butter into it so they have to work to get it. You can also get a 'tin cat' which is a live trap for mice. I put them in areas the cats don't have access to like the basement. If I catch mice in one, I dump them into an empty garbage can in the barn and drop a cat in with them.
We got a cat from my SIL, Romeo was born in a barn but my SIL got him as soon as he was able to leave his mom. She said he was peeing in the house so she gave him to us. He was only about 4 months old at the time. Romeo is an excellent hunter. I don't think I have a cat that won't kill a mouse, and my barn cats kill rats, too.
An exterminator might be cheaper... if it was a one time deal. But more mice would probably move in.
Those are some adorable kittens, and I hope they stay healthy! You might want to look into worming them, too. Any chance the kittens are already trying to chase the mice?