by Judith Curry
My Georgia Tech colleagues Peter Webster, Violeta Toma and Hyemi Kim have a new paper out entitled “Were the Pakistan floods predictable?” The topic of the 2010 Pakistan floods were discussed on previous threads here and here.
The punchline of this paper is:
The European Centre for Medium Range Weather Forecasts (ECMWF) 15-day Ensemble Prediction System (EPS) is used to assess whether the rainfall over the flood affected region was predictable. A multi-year analysis shows that Pakistan rainfall is highly predictable out to 6-8 days including rainfall in the summer of 2010. We conclude that if these extended quantitative precipitation forecasts had been available in Pakistan, the high risk of flooding could have been foreseen. If these rainfall forecasts had been coupled to a hydrological model then the high risk of extensive and prolonged flooding could have anticipated and actions taken to mitigate their impact.
Reactions to the paper
Here is some of the media reactions:
Nice paper. Important result. So why should there be any controversy? Two sources of controversy have emerged:
- ECMWF didn’t pass on the forecasts to Pakistan, sort of like the UK Met Office keeping the winter seasonal forecasts secret
- Pakistan got it right, and Webster et al.’s recommendation was inappropriate.
Both of these issues are addressed below.
Making and communicating forecasts
In response to any criticism of ECMWF and also to misconceptions encountered in the WUWT comments, I would like to clear up misconceptions about actually making a forecast of an extreme weather event, communicating it, and how people make decisions (or don’t) in response to a forecast.
First, making weather forecasts is a relatively expensive undertaking. The national meteorological centers that develop the weather prediction models, assimilate all the global weather and satellite observations and make the simulations pretty much all operate within limited national budgets: the computers aren’t big enough, and there is invariably insufficient funds to hire all the personnel that is needed. In the U.S. and to a lesser extent Europe, there is a vibrant private sector that utilizes the weather model simulations to make forecasts for specific applications (media, energy companies, farmers, etc.) In South Asia, the individual countries have meteorological agencies, which make weather forecasts with 2-3 day lead times.
ECMWF is unique since it is a multinational effort with tremendous computing capacity and funds to support research into weather forecasting. ECMWF is currently the best forecast model available. ECMWF forecasts are made available to government agencies within the EU and other partner countries. ECMWF forecasts are sold to the private sector, whereas NOAA’s forecasts are made publicly available. Relatively few people outside of the EU use the ECMWF forecasts because they are expensive to purchase ($168K Euros/year for the full suite of forecast products.) Our company, CFAN, purchases the ECMWF forecast products (which costs a big chunk of our annual income), although an agreement was made with ECMWF to use the data for free to support the Bangladesh flood forecasting project. Note: ECMWF just published this statement on its website in response to this controversy.
So the ECMWF forecast products perform very well in the South Asian monsoon region. For obvious reasons, ECMWF focuses on issues of relevance to European weather. Apparently someone at ECMWF did spot very heavy rainfall for Pakistan about 5 days before the initial event. However, the model simulations are only the first element of a forecast. Model biases need to be adjusted for. There needs to be some sense of the predictability of a certain event, e.g. whether forecasts are reliable on a certain time horizon. And there needs to be familiarity with the regional meteorology. Further, interpretation of the ensemble of simulations needs to be made to produce an actual forecast (51 simulations are made with perturbed initial conditions for each forecast period). All of this is described in the Webster et al. paper.
ECMWF did not inform anyone in Pakistan since they have no connections in Pakistan. Apart from the additional effort and analysis required to make an actual forecast from the model simulations, would anyone in Pakistan have even listened if attempts had been made to communicate this information? This is a nontrivial issue.
Recall Cyclone Nargis in 2009 which caused 138,000 deaths in Myanmar. Webster’s commentary in Nature on Nargis states that Myanmar officials did not publicize or heed the warning 2 days in advance that they received from the Indian Meteorological Department, but apparently they not want the storm to interfere with a planned referndum on that date. In the aftermath of Nargis, Myanmar officials were highly resistant to aid from the west. So in order to communicate a forecast such as Nargis or the Pakistan flood, the country has to want the forecast, trust needs to be developed between the forecasters and the government officials of the country, and some mechanism needs to be in place to actually act on the forecast. Non trivial issues.
What difference could a warning of one week have made to the Pakistanis? Better decisions could have been made in terms of operating the dams and barrages. Crops already harvested could have been transported. Livestock, seed stock, and essential household items could have been evacuated in advance, not to mention people.
Webster’s efforts in implementing an operational flood forecasting scheme for Bangladesh have shown how this can work. Webster et al. recently published a paper in the Bulletin of the American Meteorological Society that describes a decade long effort to set up an operational flood forecast system for Bangladesh, and how it is being used to reduce losses. This paper is a very good read. A real success story, until you get to the end. The meager amount of funding for this was stopped after a transfer of technology to the Bangladeshis. Within a year, the people who were trained were gone and the computers were damaged, and they couldn’t make the forecasts. We have continued to provide the forecasts to Bangladesh with funding that comes out of our own pockets; however that is not a sustainable situation.
Politics and revisionism
Both in the case of Nargis and the Pakistan floods, efforts were made by the in country meteorological agencies to claim that they made the correct forecast days in advance.
There is an article “What Pakistan did right” in Foreign Policy that claims:
On July 24, the PMD [Pakistan Meteorological Department] issued a flood warning to the provincial government of Khyber-Pakhtunkhwa (KPK). Despite these increasingly severe warnings, KPK’s citizenry did not believe them. Like victims of Hurricane Katrina and other brewing natural disasters, the residents in KPK in the waters’ path did not leave their homes for safety. The inhabitants of KPK had no experience in their lifetimes that led them to believe that a flood on such a scale was possible. This is the most likely reason why the vast majority of the flood’s 1,985 victims perished in the earliest days of the flood in KPK.
On our original Pakistan post, we stated:
On June 21, the Pakistan Meteorological Department’s monsoon forecast cautioned that urban and flash flooding could occur from July to September in the north parts of the country. The initial flooding in the north was not predicted, although the subsequent downstream flooding was forecast a day or two in advance by the river routing model.
I obtained this information from the Pakistan Meteorological Department’s web page. While most of the original forecasts are not longer there, their forecast made on Jul 27 (one day before the floods) says:
Dated:- 27th July, 2010 Time:- 10 :15 hours (PST)
SUBJECT:- FLOOD SITUATION AND WEATHER FORECAST FOR NEXT 24 HOURS:
I. FLOOD SITUATION:
River Kabul at Nowshera is in Medium flood. River Indus at Kalabagh, Chashma, Taunsa, Guddu and River Chenab at Marala and Khanki are in Low flood.
II. HYDRO METEOROLOGICAL FEATURES:
(a) Yesterday’s well marked Monsoon low over north Madhya Pradesh has moved westwards and now lies over north Gujarat (India) as Low.
(b) Seasonal low lies over Balochistan and adjoining areas.
(c) Strong Monsoon current from Bay of Bengal is penetrating sub-mountain areas of Punjab and Kashmir upto 5000 feet.
Wide Spread thunderstorm/rain with isolated heavy falls is expected over Punjab, Khyber Pakhtunkhwa, eastern parts of Balochistan, Upper Sindh and Kashmir,. Scattered thunderstorm/ rain is also expected over Gilgit-Baltistan, Lower Sindh along with Sindh Makran coast.
IV. RAINFALL (MM) RECORDED DURING PAST 24 HRS (UPTO 0800 PST):
Balakot= 71, Sialkot(Airport=64, Cantt.=15 ),Gujranwala (Cantt.)=47, Karachi(Faisal=39, Airport=28, Masroor=23), Muzaffarabad=35, Mithi & Kohat=32(each), Domel=31, Shinkiari=28, Rashidabad=27, Dir=25, Tarbela=19, Nawabshah=17, Palandri & Garhidopatta=15(each), Talhatta=14, Thatta=13, Besham & Risalpur=09(each), Tanda Dam=08, Islamabad(Shamsabad=07, Saidpur & Zeropoint=03 each, Airport =Trace),Phulra, Bahawalnagar & Lower Dir=06(each), Hyderabad & Murree=05(each), Sara-i-Alamgir, Chhor, Kamra, Kotli, Saidusharif & Attock=04(each), Daggar, Chakdara & Badin = 03(each), Dratian, Rawalakot, Hattian Bala, Jassar, Kotnaina, Nauseri & Parachinar=02(each), Cherat, Oghi, Kakul Nowshera, Palku, Zafarwal, Pattan & Joharabad=01(each), Padidan, Rahim Yar Khan & Peshawar(City & Airport)= Trace(each), Lahore (Jail Road= Trace).
v. WEATHER OUTLOOK FOR 28th to 30th July-2010:
According to latest Meteorological analysis Wide Spread thunderstorm/ rain with heavy falls at isolated places is expected over most parts of the country.
In terms of “what Pakistan did right,” this Washington Post article of Aug 2 is representative of the assessment at the time, when the Pakistani government was blamed for sluggish and disorganized response to the floods. See also my post on Pakistan flood follow up, which provides source material on the weaknesses in the Pakistani’s capability for dealing with natural disasters. The author of the Foreign Policy article seems only to have interviewed Pakistani government officials, rather than actually look at what was documented on Pakistani web pages and in news articles during that period, not to mention other reports such as Kronstedt et al. (2010).
The example of revisionism by Myanmar in response to Cyclone Nargis is also instructive. The only public forecast that was made was:
Specifically, the day before landfall4, the newspaper noted that “…the severe tropical cyclone NARGIS … is forecast to cross the coast during the next 36 hours … Under the influence of this storm, rain or thundershowers will be widespread …. [regions given] … frequent squalls with rough seas will be experienced off and along the Myanmar coast. Surface wind speed in squalls may reach  mph …”.
This statement was made in spite of the fact that the World Meteorological Organization provided them with forecasts:
The World Meteorological Organisation (WMO) supplied the Department of Meteorology and Hydrology (DMH) of Myanmar with information about cyclone Nargis as early as April 27. The WMO reported that “The New Delhi Office of the India Meteorological Department started to issue RSMC advisories at 00 UTC every 3 hours on 27 April, which continued until 06 UTC on 3 May. The last advisory said “the status of Nargis is a Severe Cyclonic Storm (50 knots) about 90 km south-west of Yangon”. The first forecast of landfall was issued at 06UTC on 1 May (36 hours in advance of landfall), which said that “Nargis will cross the Myanmar coast between 16 to 18 degrees north by the night of 2 May”. At 21 UTC of 1 May, it was forecast that “the maximum wind speed at landfall is expected to be 90 knots”. At 09 UTC on 2 May, the forecast was that “Nargis will cross near 16 degree North by 12 UTC with 90 knots wind speed”.
The revisionist statement appears in this report.
Analyzing various sources and based on a synoptic analysis DMH has produced their own cyclone prediction and issued forecasts and warnings to various layers in Myanmar.
From the discussion and reviewing of the documents from DMH, it was found that the DMH has actually detected the cyclone from a very early stage of its formation and made formal notice of it on 25th. The low pressure has been identified and the formation on low pressure Low Pressure Formation over South West Bay of Myanmar was well documented on 25 May.
We have a copy of a ppt presentation by the DMH showing their forecast products (can’t find it online; available upon request). It is a hand drawn weather map and isallobaric analysis, with much rubbing out apparent. Impossible to know whether the rubbing out indicates after the fact revisionism. But there is no value added here beyond the forecast provided by the WMO and Indian Meteorological Department. And this forecast was not made public to the citizens of Myanmar as per perusal of their newspapers, until 1800 h on May 2, the day of landfall (apparently after the time of landfall; see page 9 of the report).
I can understand a government’s motives for wanting to look competent to its people and to other countries. But this recent revisionism regarding both the Pakistan floods and Cyclone Nargis is not helping these countries improve their situation with regards to reducing vulnerability to these weather disasters.
Can we improve this situation?
Every time one of these disasters happens in the developing world, the wealthier countries contribute an enormous amount of aid, and emergency assistance from national militaries. This aid is provided for humanitarian reasons, and also in an attempt to diminish the security threat acceleration associated with the event. This aid deflects resources that could be used elsewhere.
Here is one step in the right direction:
The Regional Integrated Multi-Hazard Early Warning System for Asia and Africa (RIMES) is an international and intergovernmental institution, owned and managed by its Member States, for the generation and application of early warning information. RIMES evolved from the efforts of 26 countries in Africa and Asia, in the aftermath of the 2004 Indian Ocean tsunami, to establish a regional early warning system, within a multi-hazard framework, which generates and communicates early warning information, and builds capacity to prepare for and respond to trans-boundary hazards. RIMES was established on 30 April 2009, and registered with the United Nations, under article 102 of the UN Charter, on 1 July 2009.
RIMES objective is building capacity and providing actionable warning information towards forearmed, forewarned, and resilient communities.
We are hopeful that RIMES emphasis on early warning can help ameliorate the vulnerability to weather disasters of these large populations in Asia and Africa.
Other efforts, such as those by the Asian Disaster Preparedness Center (ADPC), are working on adaptation measures, e.g. see this forthcoming Workshop in Lao on “People Centered Approach in Flood and Land Management in the Mekong Basin.”
In closing, we envision a flat world approach, whereby the best European weather models are used by top experts in monsoon meteorology to produce forecasts. Working with national meteorological organizations and local NGOs, methods of minimizing loss of life and property can be developed. Sounds simple and relatively inexpensive? Inexpensive, yes (particularly relative the billion dollar losses); simple, no. Not simple, in spite of the fact that it would seem to clearly benefit the developing countries as well as the wealthier countries