Rodents will just eat during the day if you take the feed in at night. Few chickens will mess with a full sized rat, mice they can eat. Might choke to death doing so though.
Get a treadle feeder, control the feed and you control the rats. They will leave once the colony starts to starve and...