We drink raw. $6.50/ gallon. We were spending almost as much on the lactose free milk for DH and DD1. They have no problem drinking raw milk. My mom who hasn't been able to drink milk for 30+ years, is able to drink raw milk. She's so excited that she can have milk in her cereal again.
Oh, and keeping a buck does NOT make the milk taste bad. I've had a buck running with my does before when it was time to breed and it didn't change a thing. My bucks also share a fenceline with the does, no change.
A change in the taste of milk is from improper handling (goat milk is delicate), diet, dirty equipment, or hormones from the doe.
And some does just have yucky tasting milk. It happens.
I consider us raw milk drinkers. We used to go through four gallons or so a week, but have had to cut the dairy since one of my kids might be reacting to it. It looks like we are going to have a blood allergy test to find out for sure. I would hate to eliminate the dairy and later find out it wasn't a cause of his symptoms.
If I'm not making cheese from it, then yes, we drink raw milk. It is SOOO delicious and far better than store bought milk. Before we had goats, I hated the taste of milk. Even if I was thirsty, milk could not satisfy me.