by Judith Curry
I’ve completed a revised draft of my response the to Reply to our Uncertainty Monster paper.
When I first received the Reply, my inclination was not to respond. The Reply spouted the IPCC “party line,” about which I felt that our paper raised significant concerns. Further, this came at the worst possible time in terms of my crazy travel schedule a few weeks ago. However, the editors of the journal convinced me to reply. We dashed something off rather quickly. The reviewer of the Reply and response was quite critical of our response, and further reiterated the IPCC party line. So we’ve taken the opportunity to rework our response, and I now think it is much better.
We need to resubmit our revised version by Sunday. It would be inappropriate for me at this point to mention the authors of the Reply or post their Reply, but you can get a sense of it from our response. I would appreciate your comments on this, you will definitely see the impact of our earlier traceability discussion. Thanks in advance for your comments.
Reply to X et al.’s Comment on “Climate Science and the Uncertainty Monster”
Judith A. Curry and P.J. Webster
Abstract. X et al.’s comment provides us with a further opportunity to emphasize and clarify our arguments as to why the treatment of uncertainty in the IPCC AR4 assessment regarding attribution is incomplete and arguably misleading.
We would like to thank the authors of the Comment, all of who played leadership roles in the IPCC AR4, for their interest in our paper. The authors are correct that since the Third Assessment Report, the IPCC has placed a high priority on communicating uncertainty, and perhaps their methods have been effective in communicating with the public. However, communicating uncertainty is a very different endeavor from actually characterizing and understanding uncertainty, an effort in which we feel the IPCC falls short.
Curry and Webster (2011) raise the issue of how the IPCC has actually undertaken to investigate and judge uncertainty. X et al.’s comments focus on section 4 Uncertainty in attribution of climate change of our paper (Curry and Webster, 2011), which addresses the IPCC AR4 conclusion regarding attribution: “Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.”
The text in the IPCC AR4 (chapter 9) referenced by X et al. describes general issues and methodology for investigating uncertainty on the topic of attribution, including listing uncertainty locations. In preparing our original manuscript, we read these passages and the cited references numerous times. Listing a large number of uncertainty locations, and then coming up with a ‘very likely’ likelihood statement using expert judgment in the context of a consensus building approach, is at the heart of our concern regarding the IPCC’s treatment of uncertainty.
X et al. object to our statement in the original manuscript: “Figure 9.4 of the IPCC AR4 shows that all models underestimate the amplitude of variability of periods of 40-70 years” on the basis that we do not consider the uncertainties presented in the chapter. Figure 9.4 is presented on a log-log scale, and the magnitudes of the uncertainties for both the model simulations and the observations are approximately a decade (factor of 10). Accounting for uncertainty, a more accurate statement would have been: The large uncertainties in both the observations and model simulations of the spectral amplitude of natural variability precludes a confident detection of anthropogenically forced climate change against the background of natural internal climate variability.
X et al. state: “The remaining uncertainty in our estimates of internal climate variability is discussed as one of the reasons the overall assessment has larger uncertainty than individual studies.” Translating this uncertainty in internal climate variability (among the many other sources of uncertainty) into a “very likely” likelihood assessment is exactly what was not transparent in their assessment. We most definitely “do not appreciate the level of rigor with which physically plausible non-greenhouse gas explanations of the recent climate change are explored,” for reasons that were presented in the original manuscript. It is our assessment that the types of analyses referred to and the design of the CMIP3 climate model experiments do not support a high level of confidence in the attribution; we do not repeat the argument that we made in our original manuscript. Circular reasoning regarding 20th century attribution is associated with any model whose parameters were tuned to the 20th century climate simulations and whose 20th century forcing was not chosen in an a priori manner prior to actual referenced attribution simulations.
X et al. take issue with our statement “The high likelihood of the imprecise ‘most’ seems rather meaningless.” Whereas X et al. disagree with our statement, the InterAcademy Council Review[1] of the IPCC seems to share our concern: “In the Committee’s view, assigning probabilities to imprecise statements is not an appropriate way to characterize uncertainty.” While this passage from the IAC Review was cited in our original manuscript, we repeat it here for emphasis. Assigning a ‘very likely’ likelihood to the imprecise ‘most’ is not an appropriate way to characterize uncertainty.
With regards to the issue of traceability, the IAC made the following recommendation:
“The IPCC uncertainty guidance urges authors to provide a traceable account of how authors determined what ratings to use to describe the level of scientific understanding and the likelihood that a particular outcome will occur. However, it is unclear whose judgments are reflected in the ratings that appear in the Fourth Assessment Report or how the judgments were determined. How exactly a consensus was reached regarding subjective probability distributions needs to be documented.”
Xet al.’s assertion that they provided a traceable account of their attribution statement raises the issue of exactly what is meant by IPCC’s traceability guidelines, and what kind of traceability is actually suitable for IPCC’s assessments. A general description of the method and multiple lines of evidence ‘traceable’ to published papers is not adequate for traceability of the assignment of the confidence/uncertainty statement. Traceability allows an independent person or group to trace back to understand how the result came to be and to walk through the decision process and achieve the same result. Commonly used practices in system engineering include the traceability matrix and document control. Some fields (e.g. medical science, computer science, engineering) have stringent traceability requirements, particularly for issues that are mission critical or having life and death impacts. However one size doesn’t fit all, and the level and type of traceability required are related to the complexity of the subject matter and the criticality of the final product. With regards to the IPCC assessment reports, an independent team should be able produce the same result (assessment and assignment of uncertainty/confidence level) from the same material. Traceability is fundamentally about accountability and openness.
We clearly disagree with X et al., and we believe that our position is justified by our arguments. We are not the only people that are unconvinced by the IPCC’s attribution assessment. The existence of this disagreement is not surprising given the complexity of the issue. The existence of this disagreement implies, at the very least, one of two things: 1) the IPCC AR4 has done an inadequate job in articulating the evidence for their position; or 2) the uncertainty is much greater than acknowledged by the IPCC AR4.
Acknowledgements. Comments from the Denizens of Climate Etc. judithcurry.com are greatly appreciated. Particular thanks to Steve Mosher, John Carpenter and Pekka Perila for their input on traceability.