I was recently re-watching the 2009 documentary Fat Head, mostly for nostalgia because I enjoyed it and it did me a lot of good back when I watched it circa 2010.
If you haven’t seen it and are curious, it’s available (officially, from its distributor) on YouTube. (Weirdly, it’s age-restricted so I can’t embed it.)
This was back when the documentary Super Size Me blaming McDonalds for people being fat was only five years old and people still remembered it. Fat Head was a response-documentary criticizing Super Size Me, but it actually spent more of its time discussing the lipid hypothesis (the idea that fat and especially saturated fat causes heart disease) and the problems with it. Throughout the documentary, Tom Naughton (the filmmaker and narrator) continually refers to “the experts,” by which he mostly means the people who give official advice, such as the USDA giving food recommendations or various medical organizations telling everyone to reduce their saturated fat intake as much as possible.
“Expert,” of course, ordinarily means a person who is extremely knowledgeable in a subject or very good at it. But “expert” is also a social designation for special people to whom ordinary people are supposed to defer, generally with the assumption that they are expert in the first sense. But this introduces a problem: how do you know that someone is an expert in the first sense?
The easy way to do this is to be an expert yourself. Expertise will generally be good at recognizing expertise, as well as recognizing what is not expertise. That’s great, but if you’re an expert yourself you don’t need to know who else is an expert so you can defer to them.
So what if you’re not an expert?
Well, it gets a lot harder.
You can, of course, punt the problem to someone that you trust, but that is a general solution: it works for literally every question. How do you calculate the circumference of a circle given its diameter? Ask someone you trust.
But let’s assume, for the sake of discussion, that you want to find an expert and aren’t going to just have someone else do the work. How can you do this—again, assuming that you, yourself, are not an expert?
It certainly cannot be in the same way that an expert would, that is, by evaluating how the person does what they do. There is something left, though it’s not nearly so efficient: you can see whether the person can achieve what only an expert could achieve.
In most of the places where this is possible, it’s fairly obvious. If you want to know if a man is an expert archer, you ask him to shoot at a few things which are very difficult to hit. If you want to know if a man is an expert lock pick, you ask him to pick a difficult lock.
There are some intermediate situations, which do not admit of demonstrations which only take a moment. If you want to know if a man is an expert painter, it is not practical to ask him to go to all of the trouble of painting a painting in your sight. But you can ask him to show you paintings which he has painted, and then after he shows you some impressive paintings you have only the ordinary problem of finding out whether he’s an honest man and really is the one who painted them.
But then we come to problems which are far more difficult. How can you tell if a man is an expert teacher? The only practical effect of a good teacher is a learned student. If you have access to the students to test them, you mostly can only tell in the negative—a student who obviously knows nothing—since the whole reason to seek out a teacher is to be taught. (There are exceptions for things such as being an expert in Greek but not in teaching Greek, and you want to find an excellent teacher for your child. Let us set that aside as a special case which is easier than the one we’re trying to deal with.) However, even in the best case this is not a pure evaluation of the teacher because the end results also depends upon the quality of the student. This is clear in the case of athletics. Some people have bodies which are proportioned exceedingly well for the sport and when this is married to a disposition which finds physical activities intuitive, they would come to be very good in their sport regardless of who their teacher is; an excellent teacher will make them better but a bad teacher will still make them good (unless he gets them injured).
Medicine is an interesting hybrid of this. It is possible to evaluate a trauma surgeon mostly based on results because how well one patches up a man after a knife would or a gun shot or a bear mauling does not depend very much on the constitution of the victim. It does depend on the wound, of course, but it’s not that hard to evaluate wounds based on criteria such as their rate of blood flow or the amount of the victim which is missing.
It is nowhere near as possible to evaluate an internal medicine doctor’s treatment of chronic conditions. The human body is an unbelievably complex thing—I mean that literally; most people can’t believe the complexity involved. Biology keeps on making new discoveries that things are more complex than previous believed. All of this complexity can go wrong, and there are far fewer kinds of symptoms. In short, we have no way of evaluating what is actually wrong with a patient or how bad it actually is. Not everything is fixable; how much that doesn’t get better is the fault of the doctor and how much is the fault of the disease? We have no way of knowing, certainly not for the purpose of evaluating the doctor.
So what about the kinds of experts who give health and nutrition advice?
The first thing to notice is that the time scales are not favorable. Being healthy over decades is a thing that takes decades, and that’s a really long time over which to evaluate someone’s advice in order to determine whether their advice is worth following. And we’ve also got a problem much like in evaluating internal medicine doctors: we’re talking about how to optimize an unbelievably complex system (the human body). Worse, though, is that this kind of advice is general, and the population itself varies. There’s absolutely no reason to believe that the same dietary advice is equally good advice for all members of the population. For all we know, Frenchmen do better eating baguettes than Germans do and Germans are healthier eating sausages than Frenchmen are. For all we know, there might be two brothers and one does well on pasta while the other will get fat and sick on it. At least internal medicine doctors treat individual patients; experts who give general advice on health and nutrition give the same advice to everyone. That might be fine—no one should eat uranium, for example—but it’s not obviously fine. For all we know (without be experts ourselves) universal dietary guidelines are intrinsically a bad idea that no true expert would do, just as no true fencing expert fences with reverse grip or by holding the tip and trying to thrust the hilt into his opponent.
But even if we grant the idea, for some reason, that a true expert would give general dietary advice, how do we evaluate the expertise of a particular expert giving it? The effect that we could measure would be the superior health and fitness of the people who follow this advice to what they would have had if they didn’t follow this advice.
OK, but how on earth do you measure that? How do you identify the people who follow the advice. How do you figure out how healthy they would have been had they not followed the advice?
That last part is important because it’s extremely easy for advice which does nothing to select for people who are generally superior. To give a silly but clear example: if you give advice on how to grow taller and it’s to dunk a basketball ten times a day, every day, and then measure the average height of the adherents and the average height of the non-adherents, you’ll find that the adherents are, in fact, taller. No taller than they would have been otherwise, but certainly taller than the non-adherents. Or if your advice for strength is to pick up a three hundred pound rock and carry it five hundred feet each day, you’ll certainly find that the adherents are stronger than the non-adherents, since only very strong people will even try to follow this advice. In like manner, if you recommend that people eat a pound of arugula a day, it’s quite possible that only people who are very healthy would even consider putting the stuff in their mouth given how much (if you don’t disguise its flavor with oil or sugar) it tastes like poison. (Because it is; the bitter taste of many plants come from natural pesticides they make in order to dissuade bugs from eating them. These are just poisons that have little to no effect on us since we’re mammals and not insects.)
The basic answer is that you can’t. Not to any important degree.
There’s a related issue to the question of “how can you tell if someone is an expert?” and that’s “how does someone become an expert?” It’s related because, oversimplifying, the way you become an expert is to evaluate whether you can do what an expert can do and then change what you’re doing until you can do those things. If there’s no way to evaluate whether you’re getting better at the things an expert could do, there’s no way to tell whether the things that you’re doing are making you any better, which means that there’s no way to actually become an expert. (I’ve oversimplified quite a bit; this really deserves its own blog post.)
So what does that mean for fields where it’s not possible to tell who’s an expert?
Effectively, it means that there are no experts in that field.