Goat cost? Locally, it's anywhere from $50-$300 each, with the $300 being for adult buck meat goats or pregnant female Saanens, and the $50 being for bottle-baby dairy bucklings. You best bet might be a doe and kids for maybe $250-350—but dairy is more expensive than commercial meat is more expensive than funny pet buckling Nigerians, so keep that in mind.
If you do want just pets, buy some males and castrate them. Bucks stink.
Fencing cost varies widely, depending what you have on hand and what size of pasture you use. Would you be using chainlink, or locust posts and barbed wire, or hogwire, etc?
Housing—again, it depends. I tend to look for local scrap, or houses being taken down so that I can get materials. It's a lot cheaper than Lowes. For three goats, you wouldn't really need a big building, just deep enough that the weather can't come in through the door. If you want dairy goats, your set-up would be a bit more involved, of course.