by Judith Curry
We have looked at what uncertainty means and doesn’t mean in science, how it is measured, when it can’t be measured and how that might change through research into the big questions. Above all we asked how other people can grapple constructively with advances in knowledge and changes in thinking, instead of despairing at ‘those uncertain scientists’. – Tracey Brown and Tabitha Innocent
Sense About Science has a superb document entitled Making sense of uncertainty: Why uncertainty is part of science. From the Introduction:
Uncertainty is normal currency in scientific research. Research goes on because we don’t know everything. Researchers then have to estimate how much of the picture is known and how confident we can all be that their findings tell us what’s happening or what’s going to happen. This is uncertainty.
But in public discussion scientific uncertainty is presented as a deficiency of research. We want (even expect) certainty – safety, effective public policies, useful public expenditure.
Uncertainty is seen as worrying, and even a reason to be cynical about scientific research – particularly on subjects such as climate science, the threat of disease or the prediction of natural disasters. In some discussions, uncertainty is taken by commentators to mean that anything could be true, including things that are highly unlikely or discredited, or that nothing is known.
Some clearer ideas about what researchers mean by scientific uncertainty – and where uncertainty can be measured and where it can’t – would help everyone with how to respond to the uncertainty in evidence.
Contributors to the document include Michael Hanlon, Paul Hardaker, Ed Hawkins, Ken Mylne, Tim Palmer, Lenny Smith, David Spiegelhalter, David Stainforth, and Ian Stewart.
The document has 6 sections:
- Those uncertain scientists
- So what is uncertainty for scientists?
- Predictions and models?
- Do we even need more certainty?
- Playing on uncertainty
- Delving deeper
Much of the substance presented here is very familiar to those that have followed Climate Etc.’s uncertainty series. A few tidbits that I thought were particularly well stated:
[I]t is misleading to quantify uncertainty that cannot be quantified – in these cases there is an even greater need to talk equally clearly about what researchers do not know as what they do. ‘Unknown unknowns’ cannot be identified, much less quantified, and the best approach is to recognise this.
On emotive, economically important and political subjects, uncertainty in research findings has been played up and played down according to whether people favour or object to the implications.
‘Consensus’ suggests that scientists aim to agree. This incorrectly implies that scientists try to minimise uncertainty for the sake of finding consensus. When researchers in a field assess the ‘weight of evidence’, they don’t simply mean the number of studies on a particular question, or patients per study, but how compelling the evidence is and how thoroughly alternative explanations have been looked at – a result of the scientific process and peer review of new research.
Until we understand scientific uncertainty, we risk being seduced or confused by misrepresentation and misunderstanding. Without an understanding of uncertainty amongst the public and policymakers alike, scientists will struggle to talk about uncertainty in their research and we will all find it hard to separate evidence from opinion.
Lost in translation
The Carbon Brief comments on this document in context of climate change, in a post entitled Lost in translation: Scientific uncertainty and belief in climate change. Excerpts:
‘Making Sense of Uncertainty’ calls this the “misuse of uncertainty”: the politicisation of scientific uncertainty itself to gloss over or ignore evidence. “Smoke and mirrors”, they call it. I call it bad translation, but it all boils down to the same thing.
A simple way to remedy bad translation is to adapt the communication to fit the audience. A number of excellent publications, like this article from Somerville and Hassol – which identifies areas of confusion and offers alternatives for scientists – make plausible suggestions.
But what’s of interest to us here is the mechanics of bad translation. So, as it pops up again and again, let’s take a closer look at uncertainty and likelihood.
When scientists talk about uncertainty – and other terms like probability – they refer to how likely it is that something will happen. What the audience may hear is, “we don’t know if this is right”.
For example, the IPPC says:
“[M]ost of the observed increase in global average temperatures since the mid-20thcentury isvery likely due to the observed increase in anthropogenic greenhouse gas concentrations”.
‘Very likely’ in this case means a probability of more than 90 per cent. But the way that uncertainty is presented in the media strongly indicates that the public ‘translates’ this term as a probable cause, but with significant doubt. This is an interpretation which is significantly less certain than the original intention.
Of course, misunderstanding is possible or likely, but not guaranteed. The inexactness of the transmission means that there is a reasonably large margin of possible interpretations for the audience. They will not necessarily misunderstand, but it is likely that a significant proportion of a non-scientific audience will understand uncertainty to mean doubt, as it does in the dictionary, and likelihood to indicate possibility, not probability.
Bad translation does not just mean that the content of the message is misunderstood. It also foments a secondary kind of uncertainty in the audience’s mind. This secondary uncertainty is psychological rather than scientific, and is characterised by doubt. To use the metaphor of ‘sowing the seeds of doubt’, if the badly-translated scientific uncertainty is the seed, this secondary uncertainty is the plant that grows from it.
As a final point, we will look briefly at the politicisation of uncertainty mentioned in Sense about Science’s report. In his book, ‘Understanding Uncertainty’, the statistician, Dennis Lindley, states: “uncertainty [for most people] is a personal matter: it’s not the uncertainty but your uncertainty”. This is often overlooked in communicating scientific uncertainty, and is a key element of its politicisation and “misuse”.
Much of the strength of arguments which try to undermine the areas of agreement in climate science is in their clarity, repetition of memorable phrases and their certainty about both their own rightness and their opposition’s wrongness. Confidence is persuasive.
However, green politicians are starting to play the same game. UK climate secretary Ed Davey’s speech at the Met Office last month is another example. It was loaded with emphatic statements like “The facts don’t lie, the physics is proven. Climate change is real and it is happening now”.
Davey’s speech reached a much wider audience than the Met Office, largely because of his diatribe against “absolutely wrong and really quite dangerous” misinformation. Certainly, it riled The Telegraph and the Mail enough to respond. In the process, the newspapers accidentally promoted his speech. There may well be a question over how comfortable scientists are with this approach, but this is certainty a strategy.
JC comments: In the ten’s of thousands of words that I have written here on uncertainty, this material raises a few points that I haven’t covered adequately or bear re-emphasizing.
With regards to how the ‘public’ interprets uncertainty, I am reminded of this conversation that Peter Webster had with a Bangladeshi farmer having little formal education (which Webster has used in numerous presentations):
PJW: We hope to provide you with seasonal forecasts to help you plan your agricultural activities
FARMER: That would be good.
PJW: But we will not always be correct: Perhaps 7 times out of 10.
FARMER: (after some thought): That is fine! Only God knows 100% what will happen, and he is not telling and you are not God! Right now, we guess each year and that means we are right as often as wrong. 70% means I am ahead!
The point is this. Nearly everyone has some informal understanding of the notion of probability. Public officials and particularly those in decision making positions at government agencies have at least an operational understanding of probability, likelihood and risk, and in some agencies the relevant people have a highly sophisticated understanding of probability and risk. Journalists dealing with political, policy and technical issues also arguably have an operational understanding of uncertainty and risk.
Overegging the pudding with emphatic and overconfident statements about the science (effectively minimizing uncertainty) causes a number of problems. It motivates the other side to make contradictory statements even more emphatically and confidently. And then if your understanding turns out to be incomplete and predictions are not realized, there is a public loss of confidence in your position, which can spill over to science in general.
The most worrisome problem to me is one that I have not hitherto seen discussed is the impact of overly emphatic and overconfident statements on the science itself. Such overconfident assertions take away the motivation for scientists to challenge the consensus on detection and attribution, particularly when they can expect to be called a ‘denier’ for their efforts. The consensus among scientists about attribution of climate change extends far beyond the small community of climate scientists that actively work on detection and attribution and either publish or extensively read the primary literature. This extended group may work on climate change impacts or related problems, with individuals deriving their confidence in the consensus in a second-hand way from the emphatic nature in which the consensus is portrayed.
So there is a positive feedback loop whereby overconfidence in statements about the consensus has become a self-fulfulling prophecy, at least in the short term. This is one reason why the ‘pause’ is so interesting, with recent observations not matching the expectations from climate model projections and some people projecting that pause could continue for several decades. How scientists react to this, and how partisans in the debate play this, will be interesting to watch. I suspect that there will be backlash against the overconfidence if the pause does continue.