by Judith Curry
“It’s a national embarrassment. It has resulted in large unnecessary costs for the U.S. economy and needless endangerment of our citizens. And it shouldn’t be occurring.
What am I talking about? The third rate status of numerical weather prediction in the U.S. It is a huge story, an important story, but one the media has not touched, probably from lack of familiarity with a highly technical subject. And the truth has been buried or unavailable to those not intimately involved in the U.S. weather prediction enterprise.” — Cliff Mass, UW
Cliff Mass (see also his blog) has a very provocative and insightful post at Storm Watch7 Weather Blog entitled The U.S. Has Fallen Behind in Numerical Weather Prediction: Part I. Read the whole thing, it has lots of illustrative graphics. Here are some key excerpts:
Weather forecasting today is dependent on numerical weather prediction, the numerical solution of the equations that describe the atmosphere. The technology of weather prediction has improved dramatically during the past decades as faster computers, better models, and much more data (mainly satellites) have become available.
U.S. numerical weather prediction has fallen to third or fourth place worldwide, with the clear leader in global numerical weather prediction (NWP) being the European Center for Medium Range Weather Forecasting (ECMWF). And we have also fallen behind in ensembles (using many models to give probabilistic prediction) and high-resolution operational forecasting. We used to be the world leader decades ago in numerical weather prediction: NWP began and was perfected here in the U.S. Ironically, we have the largest weather research community in the world and the largest collection of universities doing cutting-edge NWP research (like the University of Washington!). Something is very, very wrong and I will talk about some of the issues here. And our nation needs to fix it.
But to understand the problem, you have to understand the competition and the players. And let me apologize upfront for the acronyms.
In the U.S., numerical weather prediction mainly takes place at the National Weather Service’s Environmental Modeling Center (EMC), a part of NCEP (National Centers for Environmental Prediction). They run a global model (GFS) and regional models (e.g., NAM).
The Europeans banded together decades ago to form the European Center for Medium-Range Forecasting (ECMWF), which runs a very good global model. Several European countries run regional models as well.
The United Kingdom Met Office (UKMET) runs an excellent global model and regional models. So does the Canadian Meteorological Center (CMC).
There are other major global NWP centers such as the Japanese Meteorological Agency (JMA), the U.S. Navy (FNMOC), the Australian center, one in Beijing, among others. All of these centers collect worldwide data and do global NWP.
The problem is that both objective and subjective comparisons indicate that the U.S. global model is number 3 or number 4 in quality, resulting in our forecasts being noticeably inferior to the competition.
You first notice that forecasts are all getting better. That’s good. But you will notice that the most skillful forecast (closest to one) is clearly the red one…the European Center. The second best is the UKMET office. The U.S. (GFS model) is third…roughly tied with the Canadians.
I could show you a hundred of these plots, but the answers are very consistent. ECMWF is the worldwide gold standard in global prediction, with the British (UKMET) second. We are third or fourth (with the Canadians). One way to describe this, is that the ECWMF model is not only better at the short range, but has about one day of additional predictability: their 8 day forecast is about as skillful as our 7 day forecast. Another way to look at it is that with the current upward trend in skill they are 5-7 years ahead of the U.S.
Most forecasters understand the frequent superiority of the ECMWF model. If you read the NWS forecast discussion, which is available online, you will frequently read how they often depend not on the U.S. model, but the ECMWF. And during the January western WA snowstorm, it was the ECMWF model that first indicated the correct solution. Recently, I talked to the CEO of a weather/climate related firm that was moving up to Seattle. I asked them what model they were using: the U.S. GFS? He laughed, of course not…they were using the ECMWF.
A lot of U.S. firms are using the ECMWF and this is very costly, because the Europeans charge a lot to gain access to their gridded forecasts (hundreds of thousands of dollars per year). Can you imagine how many millions of dollars are being spent by U.S. companies to secure ECMWF predictions? But the cost of the inferior NWS forecasts are far greater than that, because many users cannot afford the ECMWF grids and the NWS uses their global predictions to drive the higher-resolution regional models–which are NOT duplicated by the Europeans. All of U.S. NWP is dragged down by these second-rate forecasts and the costs for the nation has to be huge, since so much of our economy is weather sensitive. Inferior NWP must be costing billions of dollars, perhaps many billions.
The question all of you must be wondering is why this bad situation exists. How did the most technologically advanced country in the world, with the largest atmospheric sciences community, end up with third-rate global weather forecasts? I believe I can tell you…in fact, I have been working on this issue for several decades (with little to show for it). Some reasons:
1. The U.S. has inadequate computer power available for numerical weather prediction. The ECMWF is running models with substantially higher resolution than ours because they have more resources available for NWP. This is simply ridiculous–the U.S. can afford the processors and disk space it would take. We are talking about millions or tens of millions of dollars at most to have the hardware we need. A part of the problem has been NWS procurement, that is not forward-leaning, using heavy metal IBM machines at very high costs.
2. The U.S. has used inferior data assimilation. A key aspect of NWP is to assimilate the observations to create a good description of the atmosphere. The European Center, the UKMET Office, and the Canadians using 4DVAR, an advanced approach that requires lots of computer power. We used an older, inferior approach (3DVAR). The Europeans have been using 4DVAR for 20 years! Right now, the U.S. is working on another advanced approach (ensemble-based data assimilation), but it is not operational yet.
3. The NWS numerical weather prediction effort has been isolated and has not taken advantage of the research community. NCEP’s Environmental Modeling Center (EMC) is well known for its isolation and “not invented here” attitude. While the European Center has lots of visitors and workshops, such things are a rarity at EMC. Interactions with the university community have been limited and EMC has been reluctant to use the models and approaches developed by the U.S. research community. (True story: some of the advances in probabilistic weather prediction at the UW has been adopted by the Canadians, while the NWS had little interest). The National Weather Service has invested very little in extramural research and when their budget is under pressure, university research is the first thing they reduce. And the U.S. NWP center has been housed in a decaying building outside of D.C.,one too small for their needs as well. (Good news… a new building should be available soon).
4. The NWS approach to weather related research has been ineffective and divided. The governmnent weather research is NOT in the NWS, but rather in NOAA. Thus, the head of the NWS and his leadership team do not have authority over folks doing research in support of his mission. This has been an extraordinarily ineffective and wasteful system, with the NOAA research teams doing work that often has a marginal benefit for the NWS.
5. Lack of leadership. This is the key issue. The folks in NCEP, NWS, and NOAA leadership have been willing to accept third-class status, providing lots of excuses, but not making the fundamental changes in organization and priority that could deal with the problem. Lack of resources for NWP is another issue…but that is a decision made by NOAA/NWS/Dept of Commerce leadership.
I should stress that I am not alone in saying these things. A blue-ribbon panel did a review of NCEP in 2009 and came to similar conclusions (found here). And these issues are frequently noted at conferences, workshops, and meetings.
JC comments
Cliff Mass is spot on. I and other members of the meteorological community have lived with this ridiculous situation for decades. Here are some insights from my personal experiences.
My company CFAN purchases the ECMWF data set, at an annual cost of 168,000 Euros. That cost is a very big fraction of our annual income, but it is so much better than NCEP’s forecasts that we don’t waste much effort using the NCEP products. In fairness, their latest version of the Climate Forecast System (CFS) is significantly improved relative to the previous version, and the forthcoming new version of the GFS (15 day) is supposed to be a significant improvement. But ECMWF and the other models are also making ongoing improvements that will continue to keep them well ahead of NCEP.
I have worked on many field observation projects and process modeling projects that have as an objective to improve the treatments of these processes in weather and climate models. Even for U.S. based projects, ECMWF provides active participation in these projects, sending employees to meetings, etc. Whereas no one from NCEP is anywhere to be seen. Over the past several decades I have visited ECMWF about a dozen times. I have visited NCEP once, to tell them about our revolutionary new hurricane forecasting methods based on ECMWF. Their response was to tell us about their big plans for the future, which we were already pretty much doing in 2007.
One of the key ingredients to the success of ECMWF is the integration of research with development of the operational forecast model. The original idea for NCEP was that the Geophysical Fluid Dynamics Laboratory at Princeton was to be the research arm of NCEP (well it wasn’t called NCEP in the old days). But this never worked, and GFDL evolved more in the direction of ocean modeling and climate modeling, and has next to nothing to do with NCEP.
Even if NCEP had more funds and a better research arm, I still don’t think there would be much of an improvement, since NCEP has as an allergy to anything ‘not invented here.’ And there is a fundamental problem with the priorities at NOAA, IMO.
I have a vague recollection that in the 1990’s there were some congressional hearings about NCEP falling behind the Europeans, and the conclusions were that NCEP was ‘good enough’, that we didn’t really need the cadillac of weather forecasting centers. The other government agencies that relied on NCEP forecasts (e.g. transportation, agriculture, defense, etc). thought the forecasts were just fine. But the economic applications of good weather and seasonal climate forecasts are rapidly growing and possibly boundless.
NOAA’s priorities have clearly been on the development of coupled climate models for the greenhouse warming problem, and not on weather and seasonal climate forecast models. This climate modeling priority for GFDL (which really has outstanding capabilities) has sapped the resources that could have been used for the development of better weather and seasonal climate forecast models, which have a far greater socioeconomic value than do century scale climate prediction models.