by Judith Curry
“Extreme Event Learning Through Serious Fun”– a completely new way of engaging with the risks of climate change impacts and how we manage them.
Consider the following complex decision making scenario, articulated in a post by John Schaar and Heather McGray entitled “Vulnerability and Adaptation to Climate Change: A Critical Policy Challenge.”
What do you do when extended droughts make your family’s traditional farming vocation harder and harder to sustain? Or when your town’s water supply is no longer sufficient for people to draw water from their wells, forcing them to buy water from private suppliers? Or when the weakening agricultural economy leads families to pull their children out of school to do household chores, as their fathers seek seasonal work farther and farther from home?
If you represent the national or local government in a developing country, you are beginning to face more climate-related questions like these, making decisions on resource allocation increasingly difficult. You always have to start with the present – to support farmers during droughts, find ways of improving water services and see how children of poor families can be protected. However, you sense that you are not dealing with temporary phenomena, but with the foreboding of longer-term change.
What if your immediate response may actually worsen people’s ability to manage such challenges over the long term? Should you plan for fundamental shifts in agricultural policies, or seek to enhance nature’s ability to provide the water that we need, or build employment generation and social protection schemes, rather than supporting systems that may no longer be sustainable? And if so, where will the funds for these investments come from?
Pablo Suarez, the associate director of programs at the Red Cross/Red Crescent Climate Center, has some interesting ideas whereby playing games can help decision makers address such complex issues. Suarez’s ideas are discussed in two articles:
Natasha Grist, Head of Research for CDKN said: “We need robust decision making to deal with today’s climatic uncertainties. These games bring players into reality, albeit simplified, of planning for the fast and slow onset disasters that the world increasingly faces, because of climate change. We can’t afford to bury our heads in the sand any more. These simulations show what better information and preparedness can and can’t do for us, in planning for difficult times ahead. And importantly, games show us that when people are involved, human error, distraction, other priorities, scientific unpredictability, and cool headedness all play their part, as they do in real life.”
Mother Jones describes a game conducted by Suarez as part of a Community-based Adaptation Conference in Hanoi, Vietnam:
Suarez broke us into teams for a game meant to depict how one determines when and how to spent money on preparing for severe weather. Here’s how the game went down.
My team had seven people, including government staffers from the Zambia’s Ministry of Health and Pakistan’s Ministry of Interior, as well as an American academic, a French aid worker, and a communications guy based in Cairo. Each of us was given a six-sided die and 10 white beans to serve as our currency for the game. Each team also got its own six-sided die. Each player represented an individual community; our teammates would play neighboring communities. The goal was to have the most beans at the end of 10 rounds.
Before we started, each of us was asked whether we wanted to pony up one of our beans to invest in some disaster preparation—filling sandbags, revamping the radio system, stocking up on food supplies, for example. Then, we rolled the team die to see what our regional weather forecast would be, on a scale of one to six. Next, each of us rolled our own die to see our local forecast. If the sum of the team and personal dice was higher than 10, you had a weather disaster on your hands. Disaster response would cost you four beans. But, if you’d elected to pay that one bean upfront for protection, you were safe—you got to keep your beans.
After a few rounds, we were presented with another option: investing in regional forecasting technology. If our team had this, we would know the number on the team die ahead of time, giving us more information to use to decide whether or not to spend a bean upfront or risk losing four. Only two teams could have the forecasting, and it went to those that bid the biggest bunch of beans. My team didn’t venture enough beans, so we were stuck without it.
A few rounds later, there was another twist. Instead of the six-sided team die, Suarez subbed in an eight-sided die. “Have you heard of climate change?” he asked. “Things are changing. It’s unpredictable—more trouble, more risks, more chance of extreme events.”
Needless to say, the game got a whole lot harder after that. We were much more likely to be flooded out, and our bean supply was dwindling. It became an exercise in bean management, and in teamwork—trying to preserve as many beans as possible while still preventing floods. And I actually got stressed out about keeping my beans, and was feeling more than a little competitive about “winning.”
Participants have to act in real-time. They have to work together. And they have to think and talk through the challenges and consequences.
“A lot of what I have to do I can do better using games,” said Suarez in an interview after the session. Their games are about different topics—dengue outbreaks, climate science, crop insurance—but all focus on “making decisions in the face of uncertainty.”
I ended up with four beans left. I didn’t go broke, but I also spent more than I needed to on preparation. I did not win the candy bar offered as the prize for the person who kept the most beans, but my village also survived the game without a major flood.
A critical issue is ‘investing in regional forecasting technology.’ The advantages to doing this depend on the time scale of the forecast. If the timescale is a few days, then the bean would have been well spent. Once you get beyond a few days, you are dealing with probabilistic forecasts; depending on the season and where you are in the world, when averaged over multiple flood events you should come out well ahead. Once the time scale goes beyond say a month, there is substantial uncertainty. And if we are talking about longer time scales (particularly climate change time scales), it is not clear whether spending a bean on regional forecast information is of any value at all.
So, can you really be 100% certain that if you spend your bean on a forecast, that you will be able to avoid losses? The issue of whether to trust/use a forecast or not is discussed in a recent post by Roger Pielke Jr. entitled ‘Hot Hands and Guaranteed Winners
The guaranteed winner scam:
The first of these dynamics is what might be called the ‘guaranteed winner scam’. It works like this: select 65,536 people and tell them that you have developed a methodology that allows for 100 per cent accurate prediction of the winner of next weekend’s big football game. You split the group of 65,536 into equal halves and send one half a guaranteed prediction of victory for one team, and the other half a guaranteed win on the other team. You have ensured that your prediction will be viewed as correct by 32,768 people. Each week you can proceed in this fashion. By the time eight weeks have gone by there will be 256 people anxiously waiting for your next week’s selection because you have demonstrated remarkable predictive capabilities, having provided them with eight perfect picks. Presumably they will now be ready to pay a handsome price for the predictions you offer in week nine.
The hot hand fallacy:
. . . the ‘hot hand fallacy’ which was coined to describe how people misinterpret random sequences, based on how they view the tendency of basketball players to be ‘streak shooters’ or have the ‘hot hand’ . The ‘hot hand fallacy’ holds that the probability in a random process of a ‘hit’ (i.e. a made basket or a successful hurricane landfall forecast) is higher after a ‘hit’ than the baseline probability.9 In other words, people often see patterns in random signals that they then use, incorrectly, to ascribe information about the future.
Pielke Jr. concludes:
The general issue is that a bigger problem than discerning legitimate from illegitimate expertise is figuring out how to use all of the legitimate expertise at our disposal. The dynamics of the “guaranteed winner scam meets the hot hand fallacy” also presents a challenge for experts themselves in interpreting the results of research in light of evolving experience. As experts are people too, they will be subject to the incentives in and obstacles to interpreting information . . .
The dominant strategies in political discourse used to deal with this situation of too much legitimate science are to argue that there is one true perspective (the argument from consensus) or that experts can be judged according to their non-expert characteristics (argument by association). My experiences over the past decade or so related to extreme events and climate change provides a great example how such strategies play out in practice, among both experts and activists.
As we have learned, neither strategy is actually a good substitute for evaluating knowledge claims and understanding that uncertainty and ignorance are often irreducible, and decisions must be made accordingly.
JC comment: I like what Suarez is doing, and I think that decision makers can learn much from this type of approach, with the caveat of over-trusting regional forecast information.
I think a game like this is needed for climate scientists, particularly those involved in the IPCC and otherwise working at the science policy interface. Tying your science to a ‘real’ decision that has $$ consequences gives you a completely different perspective, including forcing you to think about ‘what if you are wrong,’ and how you should challenge your science to develop a more objective assessment as to the confidence a decision maker should place in your science.