You need to use "hardware cloth wire" instead of chicken wire, that is the most important thing because even if you kill one weasel, there will be another to take it's place (and another and another, etc. Also, you are allowing rats and mice to come and go into your chicken pens at will by using chicken wire, so in addition to a weasel you most likely have rats & mice. They can spoil food and spread diseases to you and your flock.
When you add the hardware cloth to your pen, make sure to dig down and bury the wire at least 6" below ground level and turned outwards another 6" at the bottom. That will reduce digging by weasels & rodents. It will aslo make tunnels they try to dig in the future easier to spot because they will be forced to start digging further out from your structure.
After that, you can go around your pen looking for any tiny gaps and fill them with insulation foam and/or steel wool. Mice & rats don't like steel wool. Make sure you look everywhere, a mouse or rat can squeeze thru anything they can get thier heads through.
You can try poison for mice, but it doesn't work for rats too long because they have sophisticated social systems in which certain rats are designated "testers". The tester rat tries new foods first, and if he dies, the other rats learn to avoid the food. We had some that would throw the poison out of the roofed area they infested by pushing it until they reached the entrance and shoving it over the edge where it would scatter.
