It sounds like you're now aware that a big part of your problem may have been due to your trying to get them to go inside while there was still too much light outside. If you adjust your "putting up" by a minute each night, as days are getting longer, and the reverse in fall, you will be more closely in tune with them.
If the cats are making you nervous, they probably are the chickens, too. A cyclone fence would still allow critters with nasty claws access to the chickens through the large mesh. Quarter inch hardware cloth 24 inches wide installed around the perimeter would greatly increase security. Then you need to figure out something so air predators and climbing ones can't get over the fence. Some people stretch bird netting over the top as long as you don't have any heavy duty climbing predators like bob cats or bears or raccoons. Make sure snakes or other burrowing critters can' get under the fence. You may need to run that hardware clothe part way underground.
If the cats are making you nervous, they probably are the chickens, too. A cyclone fence would still allow critters with nasty claws access to the chickens through the large mesh. Quarter inch hardware cloth 24 inches wide installed around the perimeter would greatly increase security. Then you need to figure out something so air predators and climbing ones can't get over the fence. Some people stretch bird netting over the top as long as you don't have any heavy duty climbing predators like bob cats or bears or raccoons. Make sure snakes or other burrowing critters can' get under the fence. You may need to run that hardware clothe part way underground.