Whoa! You have a major rat infestation if you are seeing them doing this!
Removing their access to the feeder will make them desperate, so you should manage to catch a few with the conventional sprung traps. I bait mine with peanut butter and pieces of carrot and find that very effective initially but they get wise to the traps once you catch a few. Obviously you need to put the traps where the chickens and other animals/birds cannot access them. Placing a heavy cardboard or plastic box/tote over them with one end chocked up enough for a rat to run underneath and something heavy on top to stop it being knocked/blown off is a simple method or leaning a piece of plywood against a wall to create a tunnel and putting the trap in there.
Then I revert to wax poison blocks which I nail to laths of wood and place in pieces of old 3inch diameter pipe and put them up against the side of buildings....rats like to follow the line of a building when they are scuttling about at night, so these are the places to place traps and poison. I also have rat bait stations which will take either wax blocks or poisoned grain. I have yet to try a third option which is to make plaster of Paris/peanut butter balls. The idea is that you make a kind of pastry with peanut butter and flour, make little balls with it, poke a hole in them and pour a little plaster powder inside and then seal it up with peanut butter. The rats eat it and the moisture in their stomach starts the plaster working and then it sets inside them blocking their system. Again they need to be somewhere that other creatures cannot access them. Obviously the sprung trap is the most humane means of killing them but once they get wise, you have to use other options in rotation. There is another method with a bucket of soaking grain where the rats are lured up a ramp to the lip of the bucket, see/smell the grain and jump in only to drown. I've caught a couple of mice like that but no rats so far.
If you have chickens, you will have a rat problem sooner or later. As a responsible chicken owner, it's important to tackle that for the sake of your chickens, your neighbours and yourselves. It is a recurring problem, so don't expect to ever fully sort it, just aim to control it and be vigilant once you have it under control.