I agree you need to post pictures of the scat, preferably with a ruler next to it for a size comparison. We may be able to help you identify what it is, it's amazing how much you can tell from a pile of poo...
I'm wondering if maybe it is an owl or some other bird of prey, perching on the coop at night. The neighbor used to have an owl that would sit on his tractors and watch the chickens. Then again, bird poo tends to look like bird poo, even in large quantities. Although I have seen owl pellets, which is more vomit than scat, and it tends to look like dog poo with rodent skulls and fur.
Also, you should try the leghold, many animals have experience with havaheart traps or just a phobia of cages in general and this prevents them from going into a trap regardless of the bait. Fear is a great teacher and deterent, which is why relocating predators is frowned upon, since they can and often do become trap wary.
-Kim