How to Research Nutrition
And why Dr. Campbell thinks The China Study is the way to do it.
Welcome to Part 3 (of 7) of our special series about holistic nutrition and some of the research behind the Whole Food Plant Based Diet. To learn more about the series, check out the Introduction here, where I explain how this project is something very different than yet another polemic for or against veganism!
These posts are much longer and far more complex than what I usually post at The Healthy Jew. You might find them boring – or the most interesting thing you’ve read in months. (To keep things simpler, I’ve taken out the sources from the online post - you can find them in the PDF version at the end.)
If this isn’t your cup of tea, we’ll be back soon with concise, practical guides for Jewish wellness and the Land of Israel.
In the meantime, check out last week’s Mishpacha Magazine feature about my work here at Healthy Jew.
The online version doesn’t have all the pics, so here’s a PDF of the full article:
And of course don’t forget to check out my new book, Land of Health: Israel’s War for Wellness, to learn in detail how Israel is the healthy body of the Jewish people, together with practical strategies for living well during challenging times.
By the way, the ebook is now also available for a crazy cheap introductory sale price of $1.99!
Now let’s get to business…
Last week we learned about Dr. T Colin Campbell’s decade of lab experiments with rats and animal protein. In the following two posts, we’ll review his first major test on real people: the landmark China Study which he directed.
Today we’ll examine the general background of the study, which will include fascinating insights into the science of nutrition. Next week we’ll dig into the details of Dr. Campbell’s analysis of the study’s findings.
The China Study’s Methodology
A massive 1970’s government survey in China of 880 million people showed that cancer rates varied greatly throughout the country, even though most of its citizens shared the same genetic origins. The disparities were so large that the worst-hit counties had rates 100 times larger than the lowest ones. This is quite unusual: in the US, the largest disparities are of three times. When an area in Long Island was noticed to have a slightly higher breast cancer rate than the national average (20% more), tens of millions of dollars were spent to research the issue amid intense public outcry. So what’s going on in China and what lessons can it teach us about public health for the rest of the world?
To investigate the matter, a prestigious team was put together from across the globe. Chinese scientists joined forces with prominent Cornell University researchers in the first major research cooperation between China on the US. In this unusually large study, teams spread out across rural China to gather data on 367 variables from 6500 adults in sixty-five different counties.
The study was purposely done in rural areas because people there are more settled with the same lifestyle over their entire lives. They also reliably eat locally produced food. This makes the data stable: we don’t need to suspect that someone’s health outcomes result from his former region or from the place that grew his food (or the way it was transferred).
Every participant in the study gave a blood sample and filled out a detailed questionnaire. Half also gave a urine sample, and almost a third were observed over three days to see what they ate. Samples of food were collected from local marketplaces and analyzed.
Sounds pretty thorough, right? The critics already have complaints.
Chris Masterjohn noticed that the survey was taken in the autumn, so its observational component only accounts for foods in season at the time. Then Denise Minger pointed out another, more fundamental problem in the three-day observational data. In one county, Tuoli, the villagers essentially served a three-day dairy feast in honor of their esteemed visitors, skewing that county’s data so much that Dr. Campbell disregarded it for many of his analyses. (But not all of them, as we’ll see.) How are we to know, asks Minger, that the other counties too weren’t eating differently – maybe more, maybe less – because they had a team of professionals peering over their shoulders? The China Study isn’t the first time this has happened. Any dietician will tell you how food diaries are notoriously inaccurate when trying to assess someone’s regular eating habits.
Another severe deficiency of the China Study is that it offers no information on physical activity. When we get into the details, we’ll see how this affects the study’s reliability to show us anything.
Finally, all the blood collected was combined for each county to enlarge the size of the samples and enable more variables to be tested. Yet in essence, this reduced the thousands of people evaluated to sixty-five points, which isn’t a very large pool of data to analyze.
Dr. Campbell’s Findings
Putting these shortcomings aside, what did Dr. Campbell and his team find? As he reports in The China Study, they found over 8000 statistically significant correlations (i.e. those that are too strong to just be chance) between lifestyle, diet, and diseases.
Besides specific correlations, they saw that in general, the rural Chinese dietary experience was very different than the American one. The American average of protein is 15-16% of which 81% comes from animal foods, while the Chinese get 9-10% from protein of which animals contribute only 10%. In other words, while the Chinese get less than one percent of total calories from animals, the Americans get 10-11% from animals. The study also found that the Chinese consumed more overall calories, fiber, and iron, but less fat and, as we saw, less overall protein. So while Western studies compare diets rich in animals to diets very rich in animals, the China Study compared diets rich in plants to diets very rich in plants.
Variation within a study set is very important for it to be meaningful. When most participants fall approximately within the same range, the minor differences between them probably aren’t the cause of whatever health or sickness they experience. So Dr. Campbell was delighted to find that there were lots of variations from within China to study, notwithstanding the large differences between rural China and Western countries. For example, blood cholesterol doubled from low to high, fat was six times, and fiber was five times. (Strangely, The China Study doesn’t tell us the variations of animal foods intake.)
Regarding the healthfulness of animal foods, Dr. Campbell found that the huge China project confirmed the conclusions he made from his laboratory experiments:
Among the many associations that are relevant to diet and disease, so many pointed to the same finding: people who ate the most animal-based foods got the most chronic disease.
Sounds pretty straightforward, right? Well, as with the rats, this conclusion is anything but obvious.
It’s Hard to Research Nutrition
Before we look at the data, we must be aware of the inherent limitations of proof that the China Study can offer, and why Dr. Campbell had to resort to such a low bar of proof.
Why did the good scientist need to run to rural China to find out that animal foods are unfit for human consumption? Wouldn’t it be simpler and cheaper to compare local groups of omnivorous and herbivorous Americans?
The China Study deals briefly with this question by describing the unique nature of health and lifestyle sciences. When you throw a ball into the air, physics teaches that it will always come down. But if you smoke four packs a day, you only might get lung cancer. When we research health, we can’t define immutable laws of the universe. We can only identify risk factors that are linked with various diseases. Our work will always remain statistical.
The reason researchers can’t study human health in the same way as with other exact sciences is that people aren’t predictable machines, but dynamic, extraordinarily complex organisms. We’re also very different from each other, whether in lifestyle, diet, or genes. Then each aspect of of our lifestyles, diets, and genes interact with each other inside our bodies in countless intricate pathways. Therefore, it’s practically impossible to prove beyond a doubt that any single factor brings about any single disease.
Nutrition is perhaps the most extreme example. We can easily eat a dozen different foods in one day, altogether containing thousands of different molecules. Then all of those molecules interact with each other inside the body: not just as if they were all squished around together in one petri dish, but by the way we eat in reality. Which foods were eaten at which time of the day? Were they eaten together or separately? Did the person exercise before eating, or after, or not at all? What did he drink, how much of it, and when?
Taken all together, one day of eating results in literally billions of possible combinations of chemical reactions in our bodies. Some of them will promote health, others disease.
Clinical Studies
Yet there is one way of studying isolated variables in the human body: the randomized, double-blinded, controlled interventionist (or “clinical”) studies, long considered the “gold standard” of medical research. When scientists isolate got a single molecular compound (called a drug) that may help a specific malady, it’s possible to test it conclusively.
The researchers take fifty people with the condition being studied, say high blood pressure, and split them into two groups in a way that neither the patients nor the researchers know who is in which group (“double-blinded”). Then they give one group the real experimental drug, the other one a placebo (a fake sugar pill), and see what happens. If the compound cures, the experimental group will have lower blood pressure than the control one. If it’s useless, then they’ll be the same. If the experimental group has higher counts, then we’ve got a problem.
This type of trial can provide convincing proof because it isolates the single variable that we want to know about in a way that accounts for the infinite complexity of the human body – and the unconscious biases of the researchers.
But it’s quite difficult to conduct such a rigorous test for nutrition and many other lifestyle factors. Although we can split the groups randomly, it’s impossible to double-blind the participants, because everyone will know who’s eating the carrots and who isn’t. So the control group will essentially be people that are doing nothing special, while the experimental group is being told to make a conscious decision to eat this and not that.
Such an experiment will be riddled with problems.
First of all, unlike with a pill, we can’t really know that the participants are really following the researchers’ instructions. This is particularly problematic if the trial asks them to make significant changes in their eating habits for a long time, which is usually necessary because diet influences health over years and decades, not days and weeks.
Even if they stick with the prescribed regimen, how can we know that they haven’t taken up other healthy lifestyle changes along with the veggies? It’s quite possible that their health consciousness has expanded to include exercise, meditation, yoga, and who knows what else.
This is an example of what researchers called confounders, the technical term that refers to “third” factors – that is, something besides the variable being studied (#1) as the cause of a specific outcome (#2) – which can confound and confuse a study’s results, leading us to bark up the wrong tree. In our case we can’t know what caused the result: the dietary change, the accompanying exercise, the meditation, the yoga, the who knows what else, or any combination of some or all of the above.
Another serious problem results from the lack of blinding. The very fact that the experimental group knows that their diet is being studied for possible health benefits, and the researchers following them know that too, can cause a placebo effect. Reams of research show that when people get attention from esteemed scientists for participating in their research, and take specific actions – in our case, perhaps with considerable effort – to play their role in generating new science for the world and healing for themselves, the psychological process itself can be healing, at least in the short term.
All this is the nature of the subject being studied in nutrition: life itself. The brilliant solution of the well-run clinical trial is to remove the life from the experiment, allowing us to peer into their bodies on the level of the dead, isolated chemical. This is helpful when developing drugs to cure illnesses, but far more difficult when we’re trying to determine how to live in a way that can avoid disease.
Observational Studies
Because of the difficulties in conducting high-quality clinical studies on health, much of nutrition research is done with observational, or epidemiological, studies. These studies don’t tell anyone to do anything. Instead, they collect and analyze existing data, correlating habits of different groups of people or of different time periods with their corresponding disease results. The China Study is a classic example of such a research model.
Most scientists look at such experiments as a much weaker form of evidence than clinical trials. Why? Because of the built-in flaw of all epidemiological studies: correlation doesn’t prove causation. There are two reasons for this.
First of all, we can’t really know which of the linked variables is the cause and which is the effect. For example, when you see umbrellas it’s probably raining – but that doesn’t mean umbrellas cause rain. So when we notice that people with liver cancer have higher cholesterol, we can’t know if the cholesterol (which itself may be linked with dietary factors) caused the cancer – or maybe the pathological condition itself raises cholesterol.
The second weakness of observed associations is their tendency to be confounded. The classic example of this is the fact that countries with more telephone poles have more heart disease, but that doesn’t mean that the former causes the latter. Chances are that there is a third – confounding – factor which is common to those countries that have lots of telephone poles and heart disease, probably something to do with the Western lifestyle.
But just how weak is this type of research? Dr. Campbell believes that despite their shortcomings, observational studies can have immense value – when properly interpreted by qualified scientists looking for overarching patterns, not single cause and effect relationships. In his view, the China Study’s 8000 correlations can show complex relationships between diet, lifestyle, and diseases, although they won’t prove to us that a single factor causes a single outcome. Even when examining single correlations, when we see two linked variables that are biological related (unlike telephone poles and heart disease), the evidence of the observational data becomes stronger, because there is a “mechanism of action” that can explain it.
This is how the China Study will prove the WFPB diet: with the accumulative weight of multiple, biological plausible, models.
How to Study Whole People
Dr. Campbell also believes that this is the optimal way of conducting nutritional research:
Mainstream scientific thought [says that]… science is best done when investigating single – mostly known – factors in isolation. An array of largely unspecified factors doesn’t show anything, they say… I prefer the broader picture, for we are investigating the incredible complexities and subtleties of nature itself… Everything in food works together to create health or disease. The more we think that a single chemical characterizes a whole food, the more we stray into idiocy.
To investigate whole foods on whole people, suggests Dr. Campbell, we must look at the broad dietary and disease patterns of whole populations, because that will show how multiple nutrients and chemicals act together in whole foods. Therefore, an observational design – such as the China Study – is the best way to show how an array of factors cause disease.
This doesn’t mean, concedes Dr. Campbell, that the China Study can provide absolute scientific proof. But that’s not how science works in the real world. The truth is that theories are proposed and debated until everyone accepts one as probably true due the weight of evidence that backs it up. In that way, the China Study adds a lot of weight in support of the WFPB diet. For Dr. Campbell, “the study was like a flashlight illuminating a path that I had never fully seen before.”
The China Study illuminated Dr. Campbell’s path towards the WFPB diet even though not every single correlation fit neatly into the picture he was painting. It was the overall consistency of most of the evidence that made him decide that the WFPB diet is healthy while others are not.
Throughout The China Study and his published responses to critics, Dr. Campbell emphasizes this perspective on nutritional research, time and again accusing his detractors of failing to understand his basic premise. Whenever they point out specific problems with his observations, Campbell retorts that they’re looking at the complexities of life from behind the cloudy lenses of reductionism.
For example, in the book’s second edition he responds to their claim that he tries to prove causation from correlation and ignored the inherent weaknesses of observational studies:
[This is] a false allegation. I know this principle well, as I discussed in chapter two. This criticism also assumes that scientific hypotheses should always focus on very reductionist cause-effect relationships… involving specific mechanisms presumably acting independently. But this is not how nutrition, or our bodies, work. Nutritional effects involve countless nutritional “causes” acting in concert through countless mechanisms.
For some reason (that I can’t fathom), Dr. Campbell believes that proof of his knowledge of a principle shows that he follows it. But his primary point here is that the nature of nutrition indeed requires us to look beyond simple cause and effect, to work with observational data in a way that won’t be compromised by confounders or such data’s inability to distinguish cause from effect.
This line of reasoning is hard to accept. If each single observation is meaningless, how does the fact that we’re searching for a larger “whole” make a dozen dead links come to life?
This is particularly disturbing considering that Dr. Campbell is obviously well aware that we can’t use “raw” correlations without any inhibitions. As I mentioned, he himself describes how they must be carefully chosen by qualified scientists who understand the underlying biology, and then mathematically adjusted to account for confounding variables.
In fact, a large chunk of his rebuttals to Denise Minger are about this very point. In several places in her critiques, she noticed that the China Study data actually correlates plant foods such as green vegetables and wheat with heart disease. Dr. Campbell responded at length that those correlations are unadjusted and inconsistent with known biological mechanisms. Minger then shot back, quite correctly, that she was only challenging him rhetorically on the same “raw” correlations he used throughout The China Study to indict animal foods.
(By the way, it seems to me unfortunate that Dr. Campbell spent so much of his response countering what he perceived as her suggested correlations instead of replying to her more substantial challenges to his own claims.)
She underscored this point in her final review that she published ten years after her original critique:
I’ve become somewhat of a China Study nihilist. I don’t think the original study can prove, disprove, imply, suggest, hint, damn, shout, or whisper any relevant nutritional truths. Nor do I think my critique can do any of that either. Ultimately, what I did was battle Campbell’s unadjusted correlations with more unadjusted correlations. All drawn from data that was observational to begin with. This is the weakest of weaksauce. The China Study is like a ghost of nothingness that I spent many hours poking with a stick that was also made of nothingness.
Who are we supposed to trust? Is Dr. Campbell correct in his claim that whole people can and should be studied by aggregating numerous correlations to form a whole picture, or is Denise Minger right that the whole thing is a “ghost of nothingness”?
We need to take a close look at Dr. Campbell’s central claim that observational data, evaluated and adjusted by scientists like him, can open a window to view human health as a “whole,” and that the China Study indeed shows broad patterns in favor of the WFPB diet. He invited his readers to decide for themselves how convincing his claims are. Is Dr. Campbell’s flashlight a guide or a ghost?
As he outlined in his response to Minger, Dr. Campbell used six biological models to look for connections between animal protein and disease.
Next week we’ll analyze the five main ones that are discussed in his book. (The sixth, colon cancer, he relied on “minimally”, and indeed it doesn’t appear prominently in the book in the chapter about the China study.)
Here’s a PDF version of this post along with all the footnotes: