My neighbors keep their sheep in with about a 2' high welded mesh fence (not tightened, just slapped up there, sagging and swaying) and then above that is a single strand of tensioned un-electrified wire. That's it. I don't know why they stay in - there is more grass on my side of the fence... Their fence was present before my pastures were built. I am going to have to run my own fence where our property meets, because my horses will just walk over the fence that is keeping in the sheep.... So far, the horses haven't gotten close to the fenceline because of the trees/vines/blackberry bushes.
If your fence is the PVC rail, you can find electric fence insulators that mount onto the posts. I've seen them designed to handle wide tape or the braided rope. You could run a single strand of it inside your PVC fence if needed, just to encourage them to stay on the correct side....