Tag Archives: confirmation bias

Embracing Confirmation Bias

Confirmation bias is one of the great enemies of good science. Wikipedia (I know, I know) defines the phenomenon as “the tendency of people to favor information that confirms their beliefs or hypotheses.” Most of the confirmation bias that exists in science is perpetrated unconsciously. What makes it so insidious is that our brains are hardwired to reinforce existing beliefs. Unbiased thinking does not come naturally to humans. Scientific methodology is fundamentally nothing more than a set of procedures designed to overcome confirmation bias in the search for truth.

Outside of science, confirmation bias is generally ignored and often even embraced as “faith.” In the late 19th century, for example, some Christian theologians explained away the discovery of dinosaur bones by suggesting that God had planted them in the earth to test the faith of those who believed in the literal truth of the Bible’s take on history.

Rarely are scientists ever so shameless in their efforts to preserve their existing beliefs in the face of contrary evidence. Most scientists make honest attempts to account for and overcome their biases in their research. But there are exceptions, and for some reason (actually, I know the reason, which is explained here) these exceptions often involve scientists who hold strong beliefs about what people should eat.

Whether it concerns diet or anything else, confirmation bias is classically expressed through a double standard for evidence. A nutrition scientist who is in the grip of confirmation bias has very high standards for evidence that supports ways of eating that defy his preferred diet and much lower standards for evidence that validates his preferred diet. Put another way, the same kind of evidence that is judged weak when it contradicts a biased scientist’s favored dietary prescription is considered strong when it supports the doctrine of his chosen “diet cult.”

I’ll give you an example. On June 11 of this year, James Allan Davis, a runner from Cape Town, South Africa, tweeted out a link to a BBC News story about a new Harvard study indicating a link between red meat consumption and breast cancer. The next day, Tim Noakes, an exercise scientist at the University of Cape Town and a staunch advocate of a high-fat/low-carbohydrate diet, tweeted this reply: “All evidence against meat and pro veg is based on associational studies which cannot prove causation. Need better evidence.” Minutes later Noakes tweeted a second reply that included the remark, “Associational studies prove zero.”

What Noakes said is true: Every scientist learns on day one of Statistics 101 that “correlation does not equal causation.” But as a practical matter, when a strong statistical correlation exists between two phenomena, it is very often precisely because they have a causal relationship. As the saying goes, where there’s smoke, there’s fire, and evidence from associational studies frequently turns out to be the first smoky clue pointing toward a fire of causality.

In the last century, for example, scientists representing the cigarette industry continually reminded the government and the public that “associational studies prove zero” in reference to evidence of a link between tobacco smoking and lung cancer. While these bought-and-paid-for PhD’s acknowledged that tobacco smokers were much more likely than nonsmokers to develop lung cancer, they argued—fairly—that cigarettes themselves might not be to blame. It was possible, they contended, that for some reason individuals with a predisposition to develop lung cancer were attracted to smoking. Better evidence was needed, they said, knowing full well that no scientist was ever going to turn a bunch of volunteer nonsmokers into smokers and track lung cancer diagnoses within the cohort for 20 years in order to get definitive proof of causality.

Today it is universally acknowledged that cigarette smoking causes lung cancer, but not because such interventional studies were ever done. Rather, it is because the associational evidence is extremely strong, and because there is a highly plausible mechanism to explain causality—tobacco smoke goes into the lungs, after all—and because whistleblowers exposed documents revealing that tobacco industry executives themselves privately believed that tobacco smoking caused cancer despite publicly denying it.

The associations between high vegetable intake and positive health outcomes and between high red meat intake and negative health outcomes are not quite as strong as those between tobacco smoking and lung cancer, but they are significant and consistent and, what’s more, there are plausible causal mechanisms for them. For example, high intakes of red meat are associated with elevated risk of colon cancer. Research has shown that increased red meat consumption results in an almost immediate spike in DNA damage to colon cells, and DNA damage is where cancerous tumors get started. The dots are pretty well connected.

Suppose that ten reasonable people who remembered what they had learned on day one of Statistics 101 but who somehow had no preexisting beliefs about healthy eating were presented with a summary of everything science currently knows about the health outcomes associated with high levels of vegetable consumption and red meat consumption and were then asked to adopt either a high-vegetable/low-meat diet or the opposite. Probably nine of these intelligent, unbiased persons would consider the former a safer bet, if not quite a sure thing; the tenth would be Tim Noakes (i.e. not unbiased after all).

In any case, my point is that Noakes has very high standards for evidence supporting a high-vegetable/low-meat diet, which is a de facto high-carb/low-fat diet. Does he have equally high standards for evidence that appears to support a high-fat/low-carb diet? Alas, he does not.

Exactly one week after Noakes claimed via Twitter that “associational studies prove zero,” a gentleman named David Gillespie tweeted out a graph that showed changes in per capita daily sugar consumption in the United States between 1822 and 2004 alongside changes in the percentage of the U.S. population that was obese between 1882 and 2004. Both trend lines were generally upward sloping. Although everyone agrees that correlation does not equal causation, Gillespie captioned the graph with these words: “The cause of obesity in 1 simple chart” (emphasis added). And what did Tim Noakes do when he saw this tweet? He retweeted it without additional comment.

What’s funny is that the correlation between the two trends is not particularly strong. Most of the growth in per capita sugar consumption in the U.S. occurred before 1930, whereas most of the growth in the rate of obesity occurred after 1978. It is apparent that neither Gillespie nor Noakes studied the graph very closely. When you believe what you want to believe, you see what you want to see.