I have a herd of NDs. ND bucks should not be kept with the does as they are year-round breeders and will just keep breeding the does over and over, even when too young or if they just kidded. My bucks are kept separate from the does by a wire fence. They have never tainted the milk. "Goaty" milk is a result of bad handling (i.e. not properly washing udder/storing milk/etc), mineral deficiency, poor diet, and in some cases genetics.
If you are asking about milk, I'm assuming you plan to have them for that purpose. There are pros and cons to keeping your own bucks. I would say that if you only plan to keep two does for milk, it probably isn't worth it to keep two bucks or a buck and wether. However, I would suggest you do some research on studs available in your area. Number one thing is make sure the buck and any does he is allowed to breed are tested. You don't want to expose your does to diseases. You want to make sure you are using a buck with good confirmation and from good milk lines, even if you don't plan to register your goats/their kids. If you ever keep a doe kid you'll want it to have the best chance at good confirmation and milk supply. Confirmation isn't just for show goats. The confirmation of goats is meant to "build" a goat with good health and longevity. For example, if a doe has poor udder attachment it could mean that she starts to drag her udder after a few years, or her udder will sway and cause her discomfort. It will also help you to sell kids. Many buyers want to know that the breeder cares about these things rather than just breeding any random pair. It's good to try to always breed better, meaning you want your kids to be better than your dam/buck.
On a similar note, I suggest you take the time to pick the right first goats. Get the best ones you can afford. This doesn't necessarily mean you need to buy an expensive kid from show lines. Just take some time beforehand looking at pictures and learn what makes a "good" goat. I would especially try to learn what a good udder looks like. You may not be able to see all the little bits, but you should learn what good attachments and teat placement looks like. Take the time to look at the udders of the dams for any kids you are considering.