by Judith Curry
Towards providing a robust scientific basis for climate policy, the United Nations initiated a scientific consensus building process under the Intergovernmental Panel for Climate Change (IPCC), which consisted of an intergovernmental multidisciplinary panel of experts.
Over the past two decades, the IPCC has earned substantial respect and shared the Nobel Peace Prize in 2007 for informing climate policy and raising public awareness worldwide. Amidst the increasingly intense public debate on the issue of climate change and the recent challenges to the credibility of the IPCC, the consensus approach employed by the IPCC deserves further scrutiny and reflection.
The Wikipedia defines consensus as general agreement and includes group solidarity of belief. The formal process of achieving consensus requires serious treatment of the considered opinion of each group member, noting that discussion of opposing views enhances the value of the ultimate consensus.
Consensus can play a constructive role in legitimizing policy based upon scientific research. However, there is a delicate balance between the relevance to policy makers of the consensus-based assessment and the credibility of the science. A credible consensus-based assessment is based upon the legitimacy of the experts involved, transparency of the consensus-building process, and the perception of the incorruptibility of the process.
Hulme and Mahoney state that the IPCC’s consensus approach has been largely driven by the desire to communicate climate science coherently to a wide spectrum of policy users. They further state that the IPCC consensus building process is an exercise in collective judgment about subjective Bayesian likelihoods in areas of uncertain knowledge. IPCC’s consensus approach has been a source of both strength and vulnerability for the IPCC: while the IPCC consensus approach has been effective in communicating climate science to policy makers, it has marginalized dissenting voices. (note: this paragraph has been edited from the original post).
What are the pitfalls of the consensus approach? Looking forward, is the IPCC’s consensus approach still useful?
Framing and consensus
The history of the IPCC is intimately connected with the policy issue of stabilization of atmospheric greenhouse gases. In the context of the precautionary principle and the UNFCCC, the main objective of the IPCC has been to assess whether there is sufficient certainty in the science so as to trigger action to reduce greenhouse gas emissions under the precautionary principle. This objective has led to IPCC assessments to be framed around identifying anthropogenic influences on climate, environmental and socio-economic impacts of climate change, and response strategies to anthropogenically induced climate change.
Johnston provides an interesting analysis of the framing of the IPCC assessments by analogy to a legal framework for evidence. Johnston asks the question as to whether the IPCC reports should be regarded as a legal brief – intended to persuade – or as a legal memo – intended to objectively assess both sides? In its role as an assessor of the risk of dangerous anthropogenic climate change in the context of the precautionary principle, it can be argued that the IPCC assessment reports are better characterized as legal briefs. This framework is different from a legal memo strategy that would have conducted a comprehensive assessment of natural climate variability and land use change and their impacts, in addition to anthropogenic changes in atmospheric composition, with no a priori assumption of their relative roles or policy responses. The characterization of the IPCC’s strategy as a legal brief puts the IPCC in the position of bearing the burden of proof. Johnston argues that under these circumstances, the IPCC needs an independent cross-examination. The consensus approach whereby the IPCC assesses the confidence in its own case is an unconvincing strategy under the interpretation of the IPCC assessments as a legal brief.
Van der Sluijs (2010a; a book chapter unavailable on the web) argues that the framing of the relevant scientific problem to be investigated, even the choice of the scientific discipline to which it belongs, becomes a prior policy decision.
Consensus and Uncertainty
Van der Sluijs et al. (2010b) finds that the IPCC consensus strategy has a blind spot with respect to dealing with politics and scientific dissent, by underexposing scientific uncertainties and dissent that make the chosen policy vulnerable to scientific error and by limiting the political playing field. Van der Sluijs (2010a) also argues that matters on which no consensus can be reached continue to receive too little attention by the IPCC, even though this dissension can be highly policy-relevant.
Oppenheimer et al. (2007) point out the need to guard against overconfidence and argue that the IPCC consensus emphasizes expected outcomes, whereas it is equally important that policy makers understand the more extreme possibilities that consensus may exclude or downplay. They further state that “The establishment of consensus by the IPCC is no longer as critical to governments as a full exploration of uncertainty.” Gruebler and Nakicenovic (2001) opine that there is a danger that the IPCC consensus position might lead to a dismissal of uncertainty in favor of spuriously constructed expert opinion.
Consensus and bias
One source of bias in a consensus is the selection of experts to participate in the assessment. Even with ostensible efforts to include experts with a range of perspectives, the conclusions reached by the experts depends on which experts are included in the assessment. As discussed on the hurricane thread, three different assessments on the subject of hurricanes and global warming conducted within three years of each other differed significantly in their conclusions.
Another potential source of bias in the consensus building process is cognitive bias. A cognitive bias is the human tendency to draw incorrect conclusions in certain circumstances based on cognitive factors that include information-processing shortcuts (heuristics), rather than being solely evidence based. Anchoring bias describes the tendency to rely too heavily on one trait or piece of information, such as the mean. Framing bias describes the situation or issue using an approach that is too narrow. Confirmation bias interprets information in a way that confirms preconceptions. Morgan et al. (2009) (Part III) note that scientists tend to be systematically overconfident in the face of uncertainty – that is, they produce probability distributions that are much too narrow. Actual values, once they are known, often turn out to lie well outside the tails of their previous distribution.
Cognitive biases in a consensus seeking environment can produce groupthink. Groupthink is a type of thought within a group whose members try to minimize conflict and reach consensus without critically testing, analyzing, and evaluating ideas that would challenge the consensus. Members of a group may avoid promoting viewpoints outside the comfort zone of the consensus thinking.
An assessment of a confidence interval and the range of uncertainty can be much narrower for a group operating in a consensus environment than a collation of ranges collected from individual scientists. Morgan et al. (2006) (2009) elicited subjective probability distributions from 24 leading atmospheric scientists that reflects their individual judgments about radiative forcing from anthropogenic aerosols. Consensus was strongest among the experts in their assessments of the direct aerosol effect. However, the range of uncertainty that a number of experts associated with their estimates for indirect aerosol forcing was substantially larger than that suggested by either the IPCC 3rd or 4th Assessment Reports.
Consensus and credibility
The IPCC consensus is derived from the judgment of experts. Credibility is a combination expertise and trust. The credibility challenges that the IPCC has faced over the past year (e.g. the CRU emails, the Himalayan glaciers, etc.) has arguably resulted in a substantial loss of trust in the IPCC consensus (as well as in climate science more broadly). The IAC review of the IPCC has emphasized the need for transparency and traceability in the expert judgments of confidence levels. The public confidence in the consensus of experts requires the perception that the consensus-building process is incorruptible. Once lost, trust in the experts and in the consensus-building process may not be easily regained.
Why scientific consensus fails to persuade
A fascinating study by Kahan et al. recently published in the Journal of Risk Research investigated why members of the public are sharply and persistently divided on matters on which expert scientists largely agree. This excerpt from the NSF press release summarizes their findings:
[Kahan] said the more likely reason for the disparity, as supported by the research results, “is that people tend to keep a biased score of what experts believe, counting a scientist as an ‘expert’ only when that scientist agrees with the position they find culturally congenial.” Understanding this, the researchers then could draw some conclusions about why scientific consensus seems to fail to settle public policy debates when the subject is relevant to cultural positions. “It is a mistake to think ‘scientific consensus,’ of its own force, will dispel cultural polarization on issues that admit scientific investigation,” said Kahan. “The same psychological dynamics that incline people to form a particular position on climate change, nuclear power and gun control also shape their perceptions of what ‘scientific consensus’ is.” “The problem won’t be fixed by simply trying to increase trust in scientists or awareness of what scientists believe,” added Braman. “To make sure people form unbiased perceptions of what scientists are discovering, it is necessary to use communication strategies that reduce the likelihood that citizens of diverse values will find scientific findings threatening to their cultural commitments.”
A further confounding issue for the public is that there is an alternative “consensus”, the Nongovernmental Panel on Climate Change (NIPCC). Although the NIPCC was written by individuals that have ties to advocacy groups, the report contains a very long list of scientists that support the NIPCC (over the IPCC). While the NIPCC is nowhere near as well known as the IPCC, much effort has been undertaken by those that support the mainstream IPCC consensus to discredit skeptical voices including the NIPCC. The mainstream (IPCC) scientists are then accused of arrogance when they appeal to their own authority and of being intolerant of skeptical viewpoints. The public perceives the noise in the media and the conflict, and the consensus is discounted.
I think the IPCC consensus approach was valuable in the 1990’s and arguably through the third IPCC Assessment Report, in terms of sorting through and assessing the large amount of scientific research on the topic. The Nobel Peace Prize notwithstanding, questions regarding bias and the corruptibility of the IPCC’s consensus-based assessment process are of substantial concern.
I completely agree with Oppenheimer’s assessment that at this point, a complete characterization of uncertainty is more important than consensus. While I understand the policy makers’ desire for a clear message from the scientists, at this point the consensus approach being used by the IPCC doesn’t seem to be up to challenge of an accurate portrayal of the complexities of the problem and the uncertainties.
While the public may not understand the complexity of the science and may not be culturally predisposed to accept the consensus, they can certainly understand the vociferous arguments over the science that are portrayed by the media. It is my hypothesis that better characterization of uncertainty and a more realistic portrayal of confidence levels would go a long way towards reducing the “noise” and animosity portrayed in the media that fuels the public distrust of climate science and acts to stymie the policy process.
Returning to Johnston’s legal analogy, I support a comprehensive scientific assessment based on the comprehensive legal memo framework that includes the physical basis of climate variability and change and societal and ecosystem vulnerability to climate variability and change (on a regional basis). The legal memo framework requires providing evidence for and against, and a full characterization of the uncertainties. Subsequent arguments based on the legal brief model would then stake out a specific positions on policy options and their justification (based upon science, economics, politics, values, etc.), with extensive follow-on cross examination of all briefs that are presented. This framework would broaden the assessment scope and allow for a range of perspectives, providing a better informational basis for decision making on this complex issue. This framework would also place politics squarely in the realm of the policy briefs and the cross-examination, and not in the scientific assessment process itself.