Yes, mice will urinate and leave droppings in the feeders. This can pass diseases to your birds and you. They will also eat eggs and feathers. I lost most of a clutch of eggs a chicken was brooding on last year before I got wise to the rodent problem. Oddly, the nesting box the guineas took over wasn't touched, but then again I'm not sure broody guineas ever actually sleep and they're mean as hell.
I had good luck with those battery powered electrocution traps, baited with peanut butter (the cheap, sugary crap works the best). I followed some advice on how to bait them differently: don't use the little trap door for the purpose as the mice can get at them from the outside. Tape the door over or butt the trap against a solid barrier and just put the peanut butter on floor of the trap just past the "kill plate."
I trapped a dozen that way over a week and then stopped finding droppings in my feeders.