Site icon Climate Etc.

The Uncertainty Monster

curryja

by Judith Curry

Notions of uncertainty range from everyday usage in common parlance to specific definitions appearing in the philosophical and scientific literature.

Uncertainty can prevail in situations where a lot of information is available, and new information can either decrease or increase uncertainty. New knowledge of complex systems may reveal the presence of uncertainties that were previously unknown.

A sustained and systematic enquiry of how to understand and reason about uncertainty in climate science has not been undertaken by either climate researchers or philosophers. Such enquiry is paramount because of the challenges to climate science associated with the science-policy interface and its socioeconomic importance.

I first voiced my concerns about the way uncertainty is characterized in assessment reports in 2003, at a meeting of the NRC Climate Research Committee (CRC).  The CRC subsequently held a workshop on the topic of uncertainty and climate assessments (in 2004), but the emphasis was on communicating uncertainty to decision makers and role of uncertainty in decision making.  My concerns about this issue continued to grow, and expanded to include concerns about how we reason about the uncertainties in a complex system.  Last spring, I was fortunate to be invited to attend the Royal Society’s Workshop on Handling Uncertainty in Science.  This Workshop was seminal in catalyzing and crystallizing my thoughts on the subject.

My reflection on the topic of uncertainty has culminated in a manuscript entitled “Climate Science and the Uncertainty Monster”  (in collaboration with Peter Webster), which is currently winding its way through the publication review process.   Drawing on material from this paper, this post is the first in a series that explores ways to understand, characterize, and reason about uncertainty in climate science and assessments of climate science.  This post introduces the “monster”, describes a systematic taxonomy of types and levels of uncertainty, and discusses monster coping strategies at the science-policy interface.

Introducing the uncertainty monster

The “uncertainty monster” is a concept introduced by van der Sluijs (2005) in an analysis of the different ways that the scientific community responds to uncertainties that are difficult to cope with. A monster is understood as a phenomenon that at the same moment fits into two categories that were considered to be mutually excluding.   The “monster” is therefore the confusion and ambiguity associated with knowledge versus ignorance, objectivity versus subjectivity, facts versus values, prediction versus speculation, and science versus policy.  The uncertainty monster gives rise to discomfort and fear, particularly with regard to our reactions to things or situations we cannot understand or control, including the presentiment of radical unknown dangers.

Uncertainty lexicon

The lexicon of uncertainty terms that describe the nature and levels of uncertainty follows  Walker et al. (2003) and Petersen (2006).

Nature of uncertainty

The nature of uncertainty is expressed by the distinction between epistemic uncertainty and ontic uncertainy.   Epistemic uncertainty is associated with imperfections of knowledge, which may be reduced by further research and empirical investigation.  Ontic (often referred to as aleatory) uncertainty is associated with inherent variability or randomness. The distinction between these two types of uncertainty is useful in science because each entails different conclusions regarding the reducibility of uncertainty.  Ontic uncertainties are by definition irreducible, while epistemic uncertainties are in principle reducible by further research and empirical investigations.

Epistemic uncertainty of the state of the climate system includes uncertainty due to limitations of measurement devices, insufficient data, systematic error and the subjective judgments needed to assess its nature and magnitude, extrapolations and interpolations, and variability over time or space. Uncertainty in empirical quantities can also arise from disagreement among different experts about how to interpret the available evidence.  There can also be epistemic uncertainty about how a physical, chemical or biological process works.  Epistemic uncertainties in global climate models include missing or inadequately treated physical processes, uncertainty in the numerical value of physical parameters, discretization and algorithmic approximations, and uncertainty in the specification of external forcing.

Ontic uncertainty in climate science derives from the complexity of the climate system and indeterminacy of human systems. Natural internal variability of the nonlinear climate system contributes to ontic uncertainty in climate simulations.  The climate system is stochastically uncertain because of its chaotic nature, i.e. small differences in the initial conditions of a global climate model can yield very different results. Scenarios of global greenhouse gas emissions are inherently uncertain because they depend on human behaviour, e.g. the uncertainty of the future fertility rate and future economic development.  Initial condition uncertainty is partly epistemic (inadequate and incomplete observations) and partly ontic (chaos).

Level of uncertainty

A spectrum of knowledge and uncertainty exists, ranging from complete deterministic understanding to total ignorance. Total ignorance implies a deep level of uncertainty, to the extent that we do not even know that we do not know. Walker et al. (2003) characterizes the levels of uncertainty as a progression between deterministic understanding and total ignorance:  statistical uncertainty, scenario uncertainty, and recognized ignorance.

Statistical uncertainty can be described adequately in statistical terms. An example of statistical uncertainty is measurement uncertainty, which can be due to sampling error or inaccuracy or imprecision in measurements.

Scenario uncertainty implies that it is not possible to formulate the probability of occurrence of one particular outcome. A scenario is a plausible but unverifiable description of how the system and/or its driving forces may develop in the future.  Hence the use of scenarios is associated with greater uncertainty (more ignorance) than statistical uncertainty.  Scenarios may be regarded as a range of discrete possibilities, often with no a priori allocation of likelihood.  An example of scenario uncertainty of relevance to climate science is associated with future greenhouse gas emission scenarios used to force global climate models.

Recognised ignorance refers to fundamental uncertainty in the mechanisms being studied and a weak scientific basis for developing scenarios. Reducible ignorance may be resolved by conducting further research, whereas irreducible ignorance implies that research cannot improve knowledge.  An example of irreducible ignorance is what happened prior to the big bang or what is happening beyond the cone of observations defined by the speed of light.  Border with ignorance idenotes knowledge of the presence or possibility of ignorance.

Levels of uncertainty cannot be described completely without reference to Donald Rumsfeld’s infamous statement:

“[A]s we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.”

With respect to the Walker et al. classification, a known known encompasses both deterministic certainty and statistical uncertainty. The known unknowns encompass scenario uncertainty and also situations where there is some sense of the shape of the probability distribution but knowledge of key parameters is lacking.  The unknown unknown corresponds to ignorance, including border with ignorance where the unknowns are perhaps suspected.  Unknown unknowns may be the targets of frontier research or philosophical speculations, or may correspond to future circumstances or outcomes that are impossible to predict or to know when or where to look for them.

Monster coping strategies

An adaptation of van der Sluijs’ strategies of coping with the uncertainty monster at the science-policy interface is described below.

Monster hiding. Uncertainty hiding or the “never admit error” strategy can be motivated by a political agenda or because of fear that uncertain science will be judged as poor science by the outside world.  Apart from the ethical issues of monster hiding, the monster may be too big to hide and uncertainty hiding enrages the monster.

Monster exorcism. The uncertainty monster exorcist focuses on reducing the uncertainty through advocating for more research. A growing sense of the infeasibility of reducing uncertainties in global climate modeling emerged in the 1990’s, in response to the continued emergence of unforeseen complexities and sources of uncertainties. Van der Sluijs states that:  “monster-theory predicts that [reducing uncertainty] will prove to be vain in the long run: for each head of the uncertainty monster that science chops off, several new monster heads tend to pop up due to unforeseen complexities,” analogous to the Hydra beast of Greek mythology.

Monster adaptation. Monster adapters attempt to transform the monster by subjectively quantifying and simplifying the assessment of uncertainty. Monster adaptation was formalized in the IPCC AR3 and AR4 by guidelines for characterizing uncertainty in a consensus approach consisting of expert judgment in the context of a subjective Bayesian approach (Moss and Schneider 2000).

Monster detection. The first type of uncertainty detective is scientists that are challenging existing theses and working to extend knowledge frontiers.  A second type is the watchdog auditor, whose main concern is accountability, quality control and transparency of the science. A third type is the merchant of doubt, who distorts and magnifies uncertainties as an excuse for inaction for financial or ideological reasons.

Monster assimilation.  Monster assimilation is about learning to live with the monster and giving uncertainty an explicit place in the contemplation and management of environmental risks.  Assessment and communication of uncertainty and ignorance, along with extended peer communities, are essential in monster adaptation. The challenge to monster assimilation is the ever-changing nature of the monster and the birth of new monsters.

The climate uncertainty monster is too big to hide, exorcise or adapt.  In future posts I will present arguments for ascendancy of the monster detection and assimilation approaches.  Van der Sluijs argues that the climate assessment process should be open to both technical skeptics and the doubt merchants in terms of monster detection, “and that the unpleasant way in which the game is played and the mixture of valid and ungrounded criticisms that it produces is the price that has to be paid for the key advantage for quality control of the identification of weak spots in the knowledge base.”

The challenge is to open the scientific debate to a broader range of issues and a plurality of viewpoints and for politicians to justify policy choices in a context of an inherently uncertain knowledge base.

David Spiegelhalter provided the following wise words at the recent Workshop on Handling Uncertainty in Science, that can help tame the monster:

Climate science, and particularly assessments of climate science such as the IPCC, needs to do a much better job of characterizing and reasoning about uncertainty.  The events of the past year that have challenged the credibility of climate science are symptoms of an enraged uncertainty monster.

Exit mobile version