i have found that it's equally important what's on the other side of the fence. you could have a fence and do fine. then one day, someone's eating doritos on the other side and bam. that's what happened to me. with goats who had never been fed human food ever. and it wasn't even directly outside it but around a corner where they couldn't even see. but they went crazy from the smell i guess. and went looking til they found them some doritos. you can't eat just one. oh, that's potato chips.
the other time i've had some issues was with a billy who could see the does on the other side. he put a hole in the fence that he couldn't get through but a dog could. which brings me to the concept that your billies shouldn't be able to see your does.
last thing, how old goats are to market depends on the culture in your area. with a strong latino culture, one month is popular here for cabrito.