Here's some things that may help.
I've eaten mule deer my whole life and quite a few white tail. I prefer mule deer, my wife prefers white tail, and we both like elk. I have come to respect the fact that not everyone has similar preferences when it comes to meat. However there have been some mule deer that have tasted better than others and I'll try to describe why I think some taste better than others.
I don't know if it has as much to do with what the deer are eating as how they are eating. The best mule deer I have eaten were on the gain, and I think that is more important than what they are eating. Most were killed on early seasons in September or early October, some were in the low country eating bitter brush, others in the high country eating forbes, but all of them were putting on weight.
I had some meat science classes in college, and in all of those classes it was shown that the best meat flavor came from animals that were at a high rate of gain. The further you get into the rut the higher likelihood that a buck is no longer gaining weight and may even be losing it. The same goes for deer in the late season eating sage, most of the time at that point they are maintaining, not gaining.
One other thing I have noticed is that when I started skinning and boning all my mule deer out in the field, it really improved the flavor.