‘Consensus’ by exhaustion

by Judith Curry

Regarding the consensus-seeking process for the IPCC SAR:

It is diplomacy by exhaustion. And then it becomes consensus by exhaustion as we shall see. – Bernie Lewin

At Enthusiasm, Skepticism and Science, Bernie Lewin has completed his 5 part series on Madrid 1995: The last day of climate science.  Part 5 is [here].   If you have ever wondered about the sausage making that goes into the IPCC consensus process, read the entire series.  Lewin lays out a comprehensive and well documented analysis of the issues surrounding detection and attribution in the IPCC SAR.

I’m not going to make any attempt to summarize these posts by providing excerpts here, the context and overall chain of events is important, and I encourage you to read Lewin’s articles.  Here I provide some commentary on two key topics that are of particular interest to me:  detection and attribution and the perils of an explicit scientific consensus seeking process.

Detection and attribution

A paper by Barnett et al. was of primary importance to the SAR deliberations.  From the abstract:

Estimates of the spectrum of natural variability are critical to the problem of detecting an anthropo genic signal in global climate observations. Without such information it is impossible to say that current climate change is different or unique from changes that have happened in the past and, therefore, potentially due to man-induced causes. We have estimated the spectrum of natural variability from a globally distributed set of palaeo-temperature proxies and compared it with comparable estimates from two long control integrations of coupled general circulation models – the type used to predict anthropogenic change due to greenhouse gases. None of the three estimates of the natural variability spectrum agree with each other on the low-frequency, near-global time/space scales. Until this dichotomy is resolved, it will be hard to say, with confidence, that an anthropogenic climate signal has or has not been detected.

Lewin writes:

The push for positive attribution had always been vigorously challenged within the research community, and Asheville was no exception. There it was met with ‘spirited’ debate, and even after the progress to a bottom line statement, still its retention remained tenuous. At that stage in proceedings it was not only about winning over the room. There was the legacy of the already drafted chapter to deal with. The version of the chapter they were working with was the one sent out to hundreds of scientific experts and government delegates [18Apr95]. In this version, the powerful criticisms emanating from the Barnett et al investigation was not only powerfully re-stated in the conclusion but also fully integrated into every section of the Chapter. Even if every sailor onboard were to agreed, it was almost too late to turn this ship around. But they did not agree. While we know this, and that there were extensive heated debates at Asheville, precious little detail has yet been obtained. One hint to what was going on came when the controversy broke the following spring.

So how and why did this happen in the IPCC assessments?  There is a combination of political spiking (which Lewin describes in detail) and scientific bias generated by the consensus building process itself.  I address this latter issue in my draft paper No consensus on consensus, in the section entitled Consensus and bias.  An excerpt:

Kelly (2008) provides some insight into confirmation bias, arguing that “a belief held at earlier times can skew the total evidence that is available at later times, via characteristic biasing mechanisms, in a direction that is favorable to itself.” 

Kelly (2005) describes an additional source of confirmation bias in the consensus building process: “As more and more peers weigh in on a given issue, the proportion of the total evidence which consists of higher order psychological evidence [of what other people believe] increases, and the proportion of the total evidence which consists of first order evidence decreases . . . At some point, when the number of peers grows large enough, the higher order psychological evidence will swamp the first order evidence into virtual insignificance.” Kelly (2005) concludes: “Over time, this invisible hand process tends to bestow a certain competitive advantage to our prior beliefs with respect to confirmation and disconfirmation.”

Believing what other people believe

Kelly’s idea of the higher order psychological evidence of the ‘consensus’ swamping the first order evidence seems particularly apt.  The IPCC’s detection and attribution argument is at the heart of the debate.  I would like to propose a little survey for scientists (climate and otherwise) and other denizens to assess why you believe (or do not believe) in the IPCC’s attribution statement.

1. I have read the IPCC AR4 WGI report in its entirety  YES   NO
2. I have an understanding of paleoclimate proxies in the sense that I have read at least 6 journal articles on this topic and am aware of the arguments that have criticized interpretations of these proxies  YES NO
3. I have a first order understanding of how global climate models work, in the sense that I have contributed to some aspect of model development, have conducted experiments using a global or regional climate model, and/or have read the manual for one of the global climate models or some of the primarily literature describing the construction and validation of climate models. YES NO
4. I am aware of the uncertainties surrounding the external forcing for 20th century climate model simulations (e.g. solar, volcanoes, aerosol etc) and have read at least some journal articles on this topic.  YES NO
5. I have read at least 6 papers on the subject of natural internal variability and am aware of the controversies surrounding the LIA and MWP.  YES NO

If you answer YES to at least 3 of these, I would argue that you are sufficiently informed to make an independent judgement rather than parrot the consensus.  Prior to 2009, I would have answered NO to #2, and a borderline NO on #4.  I am now up to speed on #4, and more informed but somewhat shaky on #2.    I suspect that there are many climate scientists (as classified by Anderegg et al.) that do not pass this test.  I think this is a much better test than what Anderegg et al. used, whereby someone who published 20 papers on the impact of climate change on ecosystems qualifies as having a legitimate expert opinion on the detection and attribution problem.

Verdict on ‘consensus’

I suspect that any scientist who reads Lewin’s series will be concerned about the politicization of the consensus seeking process.  In summary, I’ll close with the final paragraph from the submitted version of my No consensus on consensus paper:

The climate community has worked for more than 20 years to establish a scientific consensus on anthropogenic climate change.  The IPCC consensus building process arguably played a useful role in the early synthesis of the scientific knowledge and in building political will to act. We have presented perspectives from multiple disciplines that support the inference that the scientific consensus seeking process used by the IPCC has had the unintended consequence of introducing biases into the both the science and related decision making processes. The IPCC scientific consensus has become convoluted with consensus decision making through a ‘speaking consensus to power’ approach.  The growing implications of the messy wickedness of the climate change problem have become increasingly apparent, highlighting the inadequacies of the ‘consensus to power’ approach for decision making on the complex issues associated with climate change. Further, research from the field of science and technology studies are finding that manufacturing a consensus in the context of the IPCC has acted to hyper-politicize the scientific and policy debates, to the detriment of both.  Arguments are increasingly being made to abandon the scientific consensus seeking approach in favor of open debate of the arguments themselves and discussion of a broad range of policy options that stimulate local and regional solutions to the multifaceted and interrelated issues of climate change, land use, resource management, cost effective clean energy solutions, and developing technologies to expand energy access efficiently.