Don't get just one. Goats are herd animals and they are much, much happier with a friend. Two or three is better.
I don't consider goats expensive to care for at all. Mine eat pasture about 9 months out of the year where I live so I don't buy much hay. And no grain since mine are not milking or raising babies.
Not hard to care for. Feed and water them and keep their pens clean. Trim their hooves as needed. That's probably the hardest thing to learn to do and even that isn't hard. Helps if you have somebody who can show you how.
I deworm mine quarterly and vaccinate every two years.
They do need a good shed where they can get in out of the weather and you need really good, strong, sturdy fencing. They really are escape artists! They will go over, under, around or through it if there is even a remote possibility of doing so.
As far as what breed depends on what you want to do with them. Are they to be pets, do you want dairy goats, meat goats? Just about any goat can be a good pet if that's what you are looking for.