How Experts* Keep Failing Us—And How to Know When Not to Trust Them
David H. Freedman
New York: Little, Brown and Company, 2010, 295 pp.
Freedman is a science and business journalist and the author of A Perfect Mess and other books. After building a pretty strong case that we can’t trust the experts (scientists, finance wizards, doctors, celebrity CEO’s, high profile consultants and others), he struggles to help us know when to doubt and when to trust. The notes may sound like exaggeration, but his research supports much of what he says (if you can believe a researcher. He addresses the believability of his book in an appendix. dlm)
The bulk of the book consists of examples of expertise gone wrong in medicine, science, finance, child rearing, government, sports, and entertainment, and how and why it happens. “The fact is, expert wisdom usually turns out to be at best highly contested and ephemeral, and at worst flat-out wrong.” (7) “Perhaps a reasonable model for expert advice is … ‘punctuated wrongness’—that is, experts usually mislead us, but every once in a while they come up with truly helpful advice.” (10)
1. Some Expert Observations
“But underlying these often authoritative- and confident-seeming conclusions is a rat’s nest of confusion and misdirection largely stemming from one big question: what do you measure? (21)
“On what do Dr. Phil and Oprah base their widely heeded advice about the way we should lead our lives, other than on experience, intuition, and common sense?” “When medical researchers and other scientists lack data, they’re generally out of business. Other sorts of experts are free to—or in some cases are simply forced to—forge bravely forward, shooting from the hip.” (28)
Some traps for experts: bias, corruption, irrational thinking, pandering to the audience, ineptitude, lack of oversight, and automaticity.
These include doctors ordering unnecessary tests, sometimes from labs in which they are investors. Many informal experts advance exotic, logic-defying, hard-evidence-free ideas, especially in finance. Diet gimmicks are mostly proven through the experience of millions not to work. We seem to have an insatiable appetite for gimmicks that claim to make it easy to lose weight. In spite of thousands of tests, most performance-enhancing drugs go undetected. Law enforcement took 463 children from a cult in Texas on the basis that 9% of the children had experienced a bone fracture. Pediatrician estimates of population-wide bone fractures among all children run up to 50%.
“The simple fact is that most informal experts can spew out conclusions without much fear of being intercepted by wiser or more careful parties. Who’s filtering the recommendations of investment gurus? … Who’s going to poke your car mechanic on the shoulder and tell him that he’s replacing a perfectly good fuel injector? …in the short run, most informal experts can get away with quite a bit, and do all the time.” (34)
2. The Trouble with Scientists, Part 1.
“If we knew what we were doing, it wouldn’t be called research, would it?” (37, quoting Einstein)
“…Researchers routinely rely on flawed evidence in coming to their conclusions and in working to convince us that those conclusions are right. To put it another way, scientists are often deceptively sloppy in making and analyzing measurements.” (39)
Some ways they go wrong: measuring what doesn’t matter, mismeasuring, tossing out inconvenient data, moving the goalposts, being confounded, juggling the numbers, getting paid to get it wrong. [These sound quite serious and I’m sure you instinctively doubt the above, but just to be fair, read his examples. dlm]
In some fields scientists almost always end up looking under the streetlight (named after the joke of looking where the light is brighter instead of where you think you lost your keys). This means indirect measurements, measuring something you can measure as an indicator of what you really want to know. Such surrogate measurements often tend to lead researchers awry. Sometimes they can get really creative, especially in studying human behavior.
In medicine, studying groups that do not represent the population or projecting results from animals to humans often leads experts to make enthusiastic pronouncements that don’t work out.
Even in the best of studies of human populations there are so many interconnected variables in people’s lives that it is difficult to determine that one factor is the main cause of some behavior, condition or achievement. Multiple studies with conflicting results raise many questions about cause and effect.
“It isn’t just fancy analytical footwork that distorts study conclusions—sometimes it’s more like slipping on a banana peel.” (64) “But there’s just no getting around it: statistically speaking, being on the payroll of a company cranks up the risk of gamed study results.” “And the problem may be worse than it looks, because companies often disguise that they are behind certain findings by paying university researchers to put their names on studies actually conducted and written up by the companies themselves—so-called ghost authorship.” (65)
3. The Certainty Principle
There are reasons why we are often very much attracted to an answer that is wrong. For example, we like and tend to seek out medical advice that is definite. It simplifies our options and thinking. However, medical advice very often should be qualified.
Often expert advice that sounds nearly epiphanic when you hear it may fall apart on close inspection. So the management guru confidently says to set aside an hour a day to return phone calls, which sounds great until you realize you have no control over when other people are available to talk on the phone!
Advice sounds appealing when it seems simple and clear-cut, confident, universal, upbeat, actionable, palatable, when it make dramatic claims and is accompanied by a compelling story, when it is supported by numbers, and when it promises to prevent something that has already happened (shutting the barn door after the horse is out).
We look for the twelve steps, the seven habits, or the one-step recipe. We like a clear right answer from someone full of confidence. If it’s one-size-fits-all, we don’t have to evaluate all the options. We prefer advice that is positive. If it’s negative we may simply refuse to believe it (like the link between smoking and lung cancer). If it tells us clearly what to do in a simple formula, we like it (You can make friends by simply smiling and listening.). If it fits in with our current biases and prejudices and validates what we already believe, we like it.
“We happen to be complex creatures living in a complex world, so why would we expect answers to any interesting questions to be simple? … And that gives us a clue to recognizing advice that’s likely to be right or at least on the right track: it will be complex, it will come with many qualifications, and it will be highly dependent on conditions.” (81)
4. The Idiocy of Crowds
We expect groups to be better than individuals but the author says no. “Just one problem: the general effectiveness of groups, teamwork, collaboration, and consensus is largely a myth. Crowds, far from being reliably wise, turn out to be at least as good at discouraging and suppressing the production and dissemination of excellent work as highlighting it, and tend to bring some of the worst work to the top. Not only do group effects usually fail to protect us from flawed expertise but they introduce entirely new kinds of defects above and beyond what experts inflict on us. Crowds aren’t the solution to bad expert advice; they’re a big part of the problem.” (89)
Efficiency is impaired: the larger the group, the less gets done. The most effective groups are extremely small, like three people. Groups are frequently dominated by people who are belligerent, persuasive, persistent, manipulative, or forceful, swaying even those with doubts, because most of us don’t want to be the lone holdout. Once a majority opinion is formed, people are reluctant to argue against it because cooperation and agreement seem so important. “There’s a tendency for people in the field to believe in things if they’ve been told this is how it is, and they’ll see it that way even if the reality is different.” (97, quoting Christopher Gillberg, a child psychiatrist [an expert. dlm]
“Groups amplify bias, squash minority points of view, and can even overcome the correct point of view when it’s the majority view…. In most situations, truth doesn’t win out in groups.” (98, quoting Robert MacCoun, a decision-making researcher [expert]).
“The long U.S. housing bubble that burst toward the end of 2007, nearly wrecking the world’s economies and leading to the worst recession since the 1930s, is a striking example of how expert communities can nurture and maintain utterly wrong and even near-delusional thinking.” (98)
“Group successes, according to research, tend to depend on certain conditions: that a group is highly diverse, for example, and that there is little or no interaction between its members on the subject at hand.” (101)
5. The Trouble with Scientists, Part 2
“Reality is that which, when you stop believing in it, doesn’t go away.” (104, Philip k. Dick)
“Researchers need to publish impressive findings to keep their careers alive, and some seem unable to come up with those findings via honest work.” (108)
“It may seem strange to say it, but experts are rarely interested in getting at the truth, whatever it may be. What they want to do is prove that certain things are true. Which things? Well, whatever they happen to believe is true, for whatever reasons, or whatever will benefit their careers or status or funding the most.” (113)
“Francis Bacon noted in the late sixteenth century that preconceived ideas shape observation, causing people, for example, to take special notice of phenomena and measurements that confirm a belief while ignoring those that contradict it.” (114)
Why do some scientists manage to get it right so often? Some people end up with interesting, positive, and right results because they somehow manage to adopt the right biases, to correctly intuit which interesting, groundbreaking ideas are likely to hold up. (115)
6. Experts and Organizations
There are counterproductive sides to the faddish nature of management advice, including the retraining, meetings, paperwork, supervision and consultant meddling to get each new program up and running. “These ideas make big claims that push entrepreneurs to swing for the fences. But it leads to cutting corners on all the other things you have to do to build a company that’s going to be successful in the long run. The short-term payoff that can accrue from following this advice can blind a company to the fact that eventually it will be punished for it.” (134, quoting Eric Goldman, director of Santa Clara University’s High Tech Law Institute)
There is one strategy for winning big: take big risks. Unfortunately this is the same strategy for losing big. (143)
“Even if you identify the right companies and study them closely, you can’t figure out how to be like them. It’s not like billiards….” (145)
7. Experts and the Media
People typically absorb expert wisdom via the mass media. “The media don’t, by and large, exist solely to tell us what’s right and true; they exist to get us to read about, watch, and listen to them, and that often means selecting and presenting expert findings in a way that is entertaining, provocative, useful sounding, and otherwise satisfyingly resonant.” (150) The media doesn’t do a good job of filtering out the bogus.
Journalism is likely to “amplify the likely wrongness of a finding by exaggerating its significance; ignoring the qualifications, limits, and uncertainties…, and cheerleading for significant lifestyle changes based on it without further investigation or any real perspective.” (155-56) “But more often the media simply draw the most resonant, provocative, and colorful—and therefore most likely to be wrong—findings from a pool of journal-published research that already has a high wrongness rate.” (158)
“But, more important, the media consistently fail to highlight how untrustworthy studies turn out to be in general.” (161)
8. The Internet and the Technology of Expertise
Many current popular books are telling us this: “Thanks to the Internet, anyone can in theory get precisely the expert advice they want and need, and anyone can in theory be the expert who provides it.” “The fact is, most of what Google returns in such a search is likely to be irrelevant or wrong.” Google’s CEO, Eric Schmidt has said that the Internet is in danger of becoming a cesspool of false and misleading information because anyone can post anything they want online without any review. Good advice tends to be swamped by a much larger array of useless, misleading, subpar stuff. (171)
“Having to sit through contradictory and often inappropriate online medical advice has become a sort of public health problem in its own right.” (173) Those who are more informed and better educated can better sort through the information while everyone else drowns in second-rate information. (175)
9. Eleven Simple Never-Fail Rules for Not Being Misled By Experts
The title of this chapter should put you on your guard. This is the type of advice everyone falls for. There are no such rules. It’s not that easy. First tip: Be wary with regard to any expert pronouncement. (215) It isn’t that you shouldn’t do what the experts say, but be wary of giant, sweeping decisions without listening to experts at all. It is generally good to follow consensus expert advice that seems well supported, is not terribly burdensome to implement, and appears to have little downside, such as “Don’t text while driving.”
Be extra wary of expert advice if it is
You might want to ignore expert advice that is mildly resonant (appeals to our common sense, makes life easier, or promises to solve a pressing problem), is provocative (turns conventional wisdom upside down), gets a lot of positive attention (perhaps has been skillfully spun), is quickly embraced by other experts (who succumb to the bandwagon), appears in a prestigious journal (no guarantee), or is backed by an expert with big credentials.
Characteristics of more trustworthy advice:
· “It’s heavy on qualifying statements.” Those who describe the limits and weak points of the research are thereby furnishing evidence for credibility.
· “It’s candid about refutational evidence.” It is open about other studies that give contrary evidence.
· “It provides some context for the research.” It provides the background.
· “It provides perspective.” It helps us understand what the information means and doesn’t mean, its limitations and relevance.
· “It includes candid, blunt comments.” It expresses the expert’s own doubts and skepticism. (220-228)
* * * * * *
Your comments and book recommendations are welcome.