Site icon Climate Etc.

The ‘hard won’ consensus

curryja

by Judith Curry

The extent to which a consensus is “hard won” can be understood to depend on the personal qualities of the participating experts.” Brent Ranalli

Bishop Hill points  to an interesting new paper:

Climate science, character, and the ‘hard won’ consensus

Brent Ranalli

Abstract.  What makes a consensus among scientists credible and convincing? This paper introduces the notion of a “hard-won” consensus and uses examples from recent debates over climate change science to show that this heuristic standard for evaluating the quality of a consensus is widely shared. The extent to  which a consensus is “hard won” can be understood to depend on the personal qualities of the participating experts; the article demonstrates the continuing utility of the norms of modern science introduced by Robert K. Merton by showing that individuals on both sides of the climate science debate rely intuitively on Mertonian ideas—interpreted in terms of character—to frame their arguments.

Citation:  Ranalli, B. Climate Science, Character, and the “Hard-Won” Consensus, Kennedy Institute of Ethics Journal Vol. 22, No. 2, 183–210.  (behind paywall)

BishopHill’s post provides several excerpts from the paper. I provide some further excerpts of arguments that I found to be of particular interest.

We may define a hard-won consensus as one that emerges only after vigorous debate and a thorough examination of the range of alternative explanations. It is one in which centrifugal tendencies are strong, and the experts are drawn into agreement only reluctantly and after careful consideration. Again, the fact that a consensus is hard won is not a guarantee that it is correct. But it provides some assurance that the scientific community has done its due diligence.

JC comment:  ‘Due diligence’ (?) (!)   Does anyone recall this terminology being used in the academic literature on the climate debate or consensus prior to Steve McIntyre’s arguments?

The failure of the IPCC to seriously explore natural  variability (particularly natural internal variability on multi-decadal and longer time scales) as an alternative explanation would seem to fail the test of ‘a thorough examination of the range of alternative explanations’, where ‘experts are drawn into agreement only reluctantly after careful consideration.’   The hockey-stick line of thinking, whereby the blade was flat, is front and center in this failure.  This does not imply that the IPCC is incorrect, but rather that their consensus was not ‘hard won’ by Ranalli’s criteria.

The next question is: how does one recognize that a consensus is hard won and not merely an artifact of groupthink, or interest, or bias? One strategy is to evaluate the quality of the dialogue among experts, to look for evidence that vigorous debate took place and that multiple hypotheses were entertained before consensus was achieved. This could be done by a layperson, but only at cost and with difficulty. It would require the layperson to monitor the emerging technical literature and/or wade through extensive archives. And there are no clear criteria for deciding whether a debate has been robust enough, whether the hypotheses debated have been imaginative enough.

Another strategy is to evaluate not the quality of the dialogue but the quality of the participants. We are adept at evaluating people (or so we think—at the very least we are accustomed to it—more on this below), and it does not take long to form an opinion of a person’s character, which makes this a feasible shorthand strategy, more feasible than monitoring and evaluating the technical discussions. It makes the problem of evaluating expert consensus tractable: judge the experts, and you have a way to draw heuristic conclusions about the confidence that should be placed in their pronouncements.

Philosophers may cringe at the lack of rigor in such a chain of reasoning, and one may point in particular to our unreliability in judging character. I do not contest the lack of rigor, but I think we should entertain this line of reasoning for two reasons. First, it is well documented that scientists evaluate each other based on personal qualities, and history of science scholarship shows that since the earliest days of the modern era scientists have been at pains to develop personae—for the benefit of both the public and their fellow practitioners—to demonstrate that they possess traits considered important or necessary for participation or excellence in the profession. Second, lack of formal rigor notwithstanding, I wish to show that this is how people actually think about issues of controversy and consensus. These are terms in which debates about the reliability of consensus is played out, debates that include both scientists and laypersons, both defenders and critics of a given scientific orthodoxy.

JC comment:  I agree that both scientists and public have been using this particular heuristic.  That this heuristic was being used was made evident by the public reaction (and the reaction of some scientists) to Climategate.  It has been argued that there was nothing in the emails that changes the actual science; however the impact was on the erosion of the legitimacy of the consensus owing to what was revealed about the participants in the consensus building.

What are the criteria for evaluating persons? Here we invoke two of the Mertonian norms. First, we expect scientists to be radically individualistic and skeptical. This trait motivates them to turn a critical eye on each others’ work and collectively to leave no stone unturned when thinking about hypotheses, counterarguments, and perspectives on a problem.

In other words, among the norms that govern our expectations for the behavior and character of scientists, the norms of skepticism and disinterestedness can help to counteract groupthink, interest, and bias and facilitate the emergence of consensus that—because it is born of a rigorous and frank examination of problems, peers, and self—may be considered “hard won.”

JC comment:  while the IPCC group undeniably works hard in preparing the assessment reports, I don’t think that their consensus qualifies as ‘hard won’ by these criteria?

A survey of the climate science debate

Most of the arguments made can be located on one of several planes introduced above: they involve the role and privileges of expertise, sources of error in expert judgments, and the character of scientific experts. The “rhetorical space” in which the debate takes place [includes the following elements:]

1. Arguments about the role and privileges of expertise

2. Arguments about sources of error in expert judgment (bias, interest, groupthink)

3. Arguments about the character of experts

JC comment:  This is the neatest summary/analysis that I’ve seen on the rhetoric of the climate debate.  The parallels between the the two sides are striking.

The article then goes on to give specific examples, some of which are more on target than others.  I excerpt below the ones that I found to be insightful:

Among the amateur skeptical community there is a hearty appetite to master the science of climate change, at least in the spirit of a whodunit. Hence the incessant demands that have been made by amateur skeptics for the data underpinning published studies and even for computer code and intermediate calculations. Those who would attribute this slavish intent to exactly reproduce the work of the professionals to lack of imagination or analytical ability  miss the point. The amateur critics are not primarily interested in replicating results via independent analysis, as a scientific peer would be. Rather, they wish to audit the results, to look for flaws, evidence of wrongdoing or incompetence.

The presumption behind an audit is that expertise holds no mystique: experts may apply seasoned judgment, but every instance of such judgment can be isolated and interrogated and should be evaluated by canons of common sense accessible to anyone.

JC comment:  Ranalli ‘gets it’, this is the first such analysis that I’ve seen in the social science literature on the climate change debate.  Usually the discussion is about ‘motivated reasoning’ of the ‘deniers.’

In addition to deconstructing expertise, another tactic taken by amateur skeptics is to appeal to the authority of experts in a neighboring discipline: most importantly, statistics. With a background in mathematics, amateur skeptic Steven McIntyre, a retired minerals consultant, has made some substantive criticisms of the statistical work of Michael Mann and other paleoclimatologists. The climate experts are vulnerable in this area since the community of professional paleoclimatologists has tended to rely on its own statistical training rather than collaborating with professional statisticians.

This suggests a more general point, which is that climate science is so complex and multidisciplinary that no one individual can possibly possess the expertise to stand in an epistemically privileged position over the whole domain. Atmospheric chemists have to take on trust, much as educated laypersons do, that the solar physicists know what they are talking about and that a consensus of oceanographers is a meaningful, hard-won consensus. For skeptical author Andrew Montford, who observes that geochemists and glacier scientists who are fully convinced of the overall argument for anthropogenic global warming nevertheless downplay the adequacy of data in their own disciplines, such considerations only magnify a skeptic’s doubts. Solomon (2010, p. 46) similarly argues that scientists who have reservations about the evidence in their own area of expertise may have fewer reservations about the big picture. Recognizing this phenomenon as a general feature of expert communities, Collins  explains it with the quip that “distance lends enchantment.” Collins and Evans elaborate: “Core-scientists are continually exposed, in case of dispute, to the counter-arguments of their fellows and, as a result, are slow to reach complete certainty about any conclusion. In general, it is those in the next ring out . . . —the nonspecialists in the scientific community—who, in the short term, reach the greatest certainty about matters scientific”.

JC comment:  The previous paragraph eloquently makes a very important point that I have tried to make numerous times.  This is Michael Kelly’s ‘invisible hand’ at work.  The WG II group are staunch defenders of the consensus, and for the most part they are distant from expertise on detection and attribution..

The following paragraph is interesting to me anyways, again I find Ranalli’s analysis to be quite insightful.

Climatologist Judith Curry has offered a set of observations on the subject of character and groupthink that needs to stand alone in this survey for two reasons. First, it is a critique of the mainstream climate community that comes from a mainstream scientist rather than a skeptic. Second, although it is primarily directed at mainstream community, it applies equally well to the skeptical community. I refer to Curry’s observations on the subject of “tribalism” in science (2009). Curry argues that under pressure from ideologically motivated attacks, scientists tend to “circle the wagons” and “point the guns outward,” making themselves less likely to give criticism from the outside a fair hearing and less likely to tolerate dissent among themselves. Curry herself enjoyed the protection and solidarity of a “tribe” for approximately a year while she and her colleagues were subjected to a smear campaign on account of research on climate and hurricanes, and she came to recognize its disadvantages and dysfunctions as well as its advantages and occasional necessity. She suggests that the scientists caught up in the Climategate affair were subject to “tribal” thinking over an extended period and that this affected their judgment and their ability to be objective in certain matters. Arguably, segments of the skeptical community that are actively striving for objectivity and truth (Steve McIntyre and some of his readers, Curry implies) are equally susceptible to a tribalism that interferes with achieving those goals.

Summary and concluding remarks

In this paper we have also gotten some insight into the dynamics of a major contemporary controversy over science and expertise. As there is so little effective dialogue between the factions in the climate science wars, their accounts of reality have diverged to the extent that they sometimes seem to be inhabiting different universes. There is value, I think, in compressing those universes together and seeing how they might be made to conform. One outcome is an appreciation of the extent of common ground they share: in particular, common standards for judging the behavior and character of scientists. Thus it would be profoundly wrong for science communicators and climate change policy advocates to assume that the bulk of climate skeptics are “antiscience.” Even though they may at times hold expertise in contempt, many skeptics have a strong commitment to the norms of science. At times this commitment seems even more puritanical than that of the scientific establishment itself: as seen, for example, in skeptics’ demands for more inclusive participation (universalism) and more open sharing of data and methods (communalism).

JC comment:  Again these are unusual (and welcome) insights from a social scientist covering the climate debate.  And I would agree with this characterization of many of the climate change skeptics.  This does not mean the skeptics are necessarily ‘right’ in their scientific conclusions, but characterizing them as ‘anti-science’ is absolutely inappropriate.

JC summary:  Ranalli has provided an even handed and insightful analysis of the climate debate.   I don’t agree with all of his statements and points; I have chosen to highlight the points that I find insightful. In any event, I would say that CAGW proponents interested in countering the strategies of skeptics would do well to understand the underlying dynamics of the debate rhetoric.

And if ‘character’ really does matter,  attempts to score points through advocacy and rhetoric in the interest of propaganda will not have the desired effect: rather, these will just contribute to the character deficit associated with the consensus in the wake of Climategate.

Exit mobile version