by Judith Curry
[W]e have a field of sort-of-science in which hypotheses are treated as facts because they’re too hard or expensive to test, and there are so many hypotheses that what journalists like to call “leading authorities” disagree with one another daily. – Gary Taubes
Why nutrition science is so confusing
Gary Taubes has an article in the NYTimes Why nutrition is so confusing, which has some interesting parallels with climate science. Excerpts:
Since the 1960s, nutrition science has been dominated by two conflicting observations. One is that we know how to eat healthy and maintain a healthy weight. The other is that the rapidly increasing rates of obesity and diabetes suggest that something about the conventional thinking is simply wrong.
In total, over 600,000 articles have been published purporting to convey some meaningful information on these conditions. It would be nice to think that this deluge of research has brought clarity to the issue. The trend data argue otherwise. The more we learn, the more we need to know.
Because the nutrition research community has failed to establish reliable, unambiguous knowledge about the environmental triggers of obesity and diabetes, it has opened the door to a diversity of opinions on the subject, of hypotheses about cause, cure and prevention, many of which cannot be refuted by the existing evidence. Everyone has a theory. The evidence doesn’t exist to say unequivocally who’s wrong.
In nutrition, the hypotheses are speculations about what foods or dietary patterns help or hinder our pursuit of a long and healthy life. The ingenious and severe attempts to refute the hypotheses are the experimental tests — the clinical trials and, to be specific, randomized controlled trials. Because the hypotheses are ultimately about what happens to us over decades, meaningful trials are prohibitively expensive and exceedingly difficult. And before any of this can even be attempted, someone’s got to pay for it. Without such trials, though, we’re only guessing whether we know the truth. [A]dvice to restrict fat or avoid saturated fat has been based on suppositions about what would have happened had such trials been done, not on the studies themselves.
Nutritionists have adjusted to this reality by accepting a lower standard of evidence on what they’ll believe to be true. The associations that emerge from these studies used to be known as “hypothesis-generating data,” based on the fact that an association tells us only that two things changed together in time, not that one caused the other. So associations generate hypotheses of causality that then have to be tested. But this hypothesis-generating caveat has been dropped over the years as researchers studying nutrition have decided that this is the best they can do.
One lesson of science, though, is that if the best you can do isn’t good enough to establish reliable knowledge, first acknowledge it — relentless honesty about what can and cannot be extrapolated from data is another core principle of science — and then do more, or do something else.
Obesity and diabetes are epidemic, and yet the only relevant fact on which relatively unambiguous data exist to support a consensus is that most of us are surely eating too much of something. We’re going to have to stop believing we know the answer, and challenge ourselves to come up with trials that do a better job of testing our beliefs.
A truly magical theory
Melanie Phillips as a post at the Electric Media Blog entitled A truly magical theory, that provides some climate science examples of what Taubes discussed in the preceding article. Excerpts:
Dame Julia Slingo, Chief Scientist at the UK Met Office, says that while there is ‘not yet definitive proof’, nevertheless ‘all the evidence’ suggests that ‘climate change’ is a contributory factor to the storms and heavy rainfall now causing the devastating floods in southern England.
Just how does Dame Julia arrive at this ‘significant’ conclusion?
The Met Office paper also says that, for various reasons (UK weather notoriously volatile, appropriate computer modelling systems not yet in place to ‘prove’ what they already know to be unarguably true) attributing changes in rainfall, regional climate and weather extremes to ‘climate change’ is, uh, ‘challenging’.
By which Dame Julia et al mean these changes cannot be attributed to AGW. But does that dent their certainty that AGW is the cause? Of course not!
‘There is no evidence to counter the premise that a warmer world will lead to more intense daily and hourly heavy rain events.’
And so the evidence to support the theory that ‘climate change’ has caused the storms is… that there’s no evidence to falsify what is merely a supposition.
Truly, AGW is a magical theory that explains absolutely everything – including diametrically contradictory phenomena, lack of logic and absence of evidence – whenever people observe profoundly, ‘Something funny’s happening to the weather’.
I have another theory to explain the current deluge. It is Galileo, Newton and Einstein weeping uncontrollably from above.
Severe testing of hypotheses
Taubes’ article mentions ‘severe testing.’ Joel Katzav has a very interesting article ‘Severe testing of climate change hypotheses’ published in History and Philosophy of Modern Physics [link] to complete manuscript. This is a very interesting paper; I provide here only some brief excerpts:
On the severe testing epistemology of science, scientific hypotheses’ truth should be assessed in light of how well they have withstood severe tests. According to Mayo’s version of the severe testing epistemology, an experimental result, e, counts as good evidence for an hypothesis h to the extent that the test that yielded e severely passes h with e. h passes a severe test with e if, and to the extent that, h fits e and it would have been very unlikely that h would have fitted test results as well as it does had h been false. Mayo’s definition makes, in accord with the severe testing approach, how good evidence is depend on the test procedure that produces the evidence. But she also wants to make sure that good evidence is strong indeed, and thus adopts a demanding version of the severe testing approach.On her view, we need a definition of ‘successful severe test’ that guarantees that (successful) severe tests are reliable indicators of the absence of error in tested hypotheses. As to evidence that is less than good evidence for acclaim, Mayo allows it an important role in identifying aspects of the claim that remain to be tested. On her view,such evidence thus helps us to see how far the claim is from the truth and what alternatives to it remain to be explored. Attaching probabilities to claims is, supposedly, neither needed nor desired in science.
From the conclusions:
As to OUR FAULT, IPCC-AR4 does not appear to have severely tested it. From Mayo’s perspective, then, IPCC-AR4 does not appear to provide good evidence for OUR FAULT. Mayo’s approach does indicate that the evidence may nevertheless provide guidance as to how far this claim may be from the truth and about what assumptions need further testing if it is to be well supported. But OUR FAULT is itself a broad estimate of where the truth may be with respect to a certain possible cause of global warming. There is no further error range estimate attached to it. Indeed, attaching such a further estimate would just amount to making another, weaker claim such as that anthropogenic greenhouse gas emissions are responsible for more than a third of the post-1950 warming.
JC note: This is a very rich paper, I have invited Katzav to do a guest post.
JC reflections
Nutrition science and climate science share some common challenges: complex system(s) and many confounding factors. Severe tests for nutrition science can in principle be done, but they are very expensive and take decades. Severe tests for climate science require better observational evidence, particularly in the past.
When there’s no evidence to falsify what is merely a supposition,we are left with “magical theories that explains absolutely everything – including diametrically contradictory phenomena, lack of logic and absence of evidence.”
I agree with Mayo/Katzav that when evidence is inadequate for a severe test, it is important to identify aspects of the hypothesis that remain to be tested, and to provide an assessment of alternative hypotheses.
In climate science, the limitations of available evidence and weak reasoning behind the high confidence levels of the IPCC conclusions reflect acceptance of lower standards of evidence on what is believed to be true.