I have had both mule deer and whitetail. Any difference in the flavor can be attributed to the diet and how the animal was killed and handled after killing. Good kills and proper processing will yield meat that does not have any gaminess. If you took a steer, gut shot it and chased it around for half a day before it died and then hauled a long way on a hot day before getting it cooled down and then cut it up the next day or two after leaving it to hang in the sunshine, you would not eat beef either.
There can be a flavor difference between a mule deer that had sagebrush as a large part of its diet and a whitetail fattened up in a cornfield. But a mule deer fattened up on a rancher's alfalfa field is every bit as good.
Sitka deer are a different animal all together water grasses are a major part of the diet for them Sitka sits on a Island Chichagof they have limited are to roam managed to clean both my active coops