You have to be careful. VERY careful.
If the milk is not properly treated, it can get contaminated. If the farmer is not anal retentive about cleanliness, you can get sick. If the cows are not cared for, bad things can happen. This all means that you have to be very careful about from where you buy it.
Go see the farm. Are things tidy, organized, or are there piles of poop everywhere, trash scattered about, and horrible smells (Don't fret junk laying around. That seems to be part of the farming experience). Is the barn clean, reasonably fresh smelling, and nicely maintained, or is it filthy, reeking of waste odors, and caked with manure? Do the animals look healthy, happy, and well cared for? Does the farmer seem to treat his animals with care? A farmer who cares about the property, animals, and his own self will care about his milk.
IMHO, the best way to get raw milk is to raise your own jersey or miniature jersey. They're a lot of work, and require a lot of time investment, but they pay back in psychological benefits, grass cutting service, and the most amazing, fresh, dairy products around. I'd sooner be without a family cow as I would without a refrigerator, heat in the house and running water.
Finally, I don't consume raw milk for health reasons. I consume it because it tastes divine. Store bought milk tastes like boiled chalk water, and I avoid it like the plague.