Definitely try backyard herds! I know around here yearling bucks/whethers get sold for dirt cheap during the fall and going into winter. It probably would be cheaper to buy a few at the end of the season and send them to butcher than it would cost to feed one until it was big enough!
If you do raise goats for meat, you would most likely have to get two. Goats do best in herd type settings, so having one, even if just for a short time, would be tough for the goat. Lonely goats can drive their people mad! (Constant baaa-ing!)
In the end, it probably would not be too difficult or expensive to raise a goat to slaughter, though if you get to talking to people you may find someone who you can just buy them off of closer to slaughter age (I admit I don't know a ton about what age you slaughter goats. I'm going off my knowledge on lamb!)
Good luck though! If you have husbandry questions feel free to ask. There are quite a few knowledgeable members on here! I'm more familiar with husbandry type issues as well.