by Judith Curry
The Weather Forecasting Improvement Act of 2013 introduced by Environment Subcommittee Vice Chairman Jim Bridenstine will prioritize the mission of NOAA to include the protection of lives and property, and make funds available to improve weather-related research, operations and computing resources.
Recently there was a hearing before the U.S. House Subcommittee on the Environment regarding The Weather Forecasting Improvement Act of 2013 with the following objective:
“To prioritize and redirect NOAA resources to a focused program of investment on near-term, affordable, and attainable advances in observational, computing, and modeling capabilities to deliver substantial improvement in weather forecasting and prediction of high impact weather events, such as tornadoes and hurricanes, and for other purposes.”
The witnesses at the hearing:
- The Honorable Kathryn Sullivan, Acting Administrator, National Oceanic and Atmospheric Administration
- Dr. Kelvin Droegemeier, Vice President for Research, Regents’ Professor for Meteorology, Weathernews Chair Emeritus, University of Oklahoma
- Dr. William Gail, Chief Technology Officer, Global Weather Corporation, President-Elect, American Meteorological Society
- Dr. Shuyi Chen, Professor, Meteorology and Physical Oceanography, Rosentiel School of Marine and Atmospheric Sciences, University of Miami
Kathryn Sullivan does a good job ‘selling’ NOAA; nothing particularly surprising in her testimony. Kelvin Drogemeier provides a good perspective on improving forecasts of high impact weather such as tornadoes and hurricanes on relatively short time scales. Here I focus on the testimonies by Shuyi Chen and Bill Gail.
The excerpts provided here are on the section of Shuyi’s testimony entitled Weather Forecasts Beyond Two Weeks: A New Frontier:
Focusing on the near-term improvement of weather forecasts and warnings, especially high impact weather such as tornadoes and hurricanes, is important. But we must also recognize that the need and economic values of extending weather forecasts beyond one week, known as extended weather forecasts on subseasonal time scale (1–4 weeks). Recent research has found that the occurrence probabilities of tornados and hurricanes significantly fluctuate on that time scale. Other phenomena also fluctuating on that time scale include drought, flooding and heat waves, all having great impacts on society. A better understanding and improvement of subseasonal forecasts are needed to bridge weather and seasonal forecasts at the weather and climate interface, which has been documented in the World Weather Research Program (WWRP) Implementation Plan10 and the NRC report (2010b).
From the end-user perspective, extended weather forecasts are very important, because many management decisions, such as in agriculture and preparation for high impact weather (flood, heat waves, hurricanes, wild fires) and proactive disaster mitigation, fall into this scale. Reliable and skillful subseasonal forecasts for this timescale would be of considerable value.
Recent research has indicated important potential sources of predictability for this time range such as slowing evolution in tropical convection, stratospheric influences, and land/ice/snow interactions. Recent improvements in computing resources and model development may make it possible to develop a better representation of these sources of extended weather predictability.
Several operational centers are now producing operational subseasonal forecasts. In principle, advanced notification, on the order of two to several weeks, of the probabilities of hurricanes, tornados, severe cold outbreaks and heat waves, torrential rains, and other potentially high impact events, could help protect life and property; humanitarian planning and response to disasters; agriculture and disease planning/control (e.g., malaria and meningitis); river-flow and river-discharge for flood prediction, hydroelectric power generation and reservoir management; landslides; coastal inundation; transport; power generation; insurance. There are tremendous potential benefits from reliable extended weather forecasts to reductions in mortality and morbidity and to economic efficiencies across a broad range of sectors.
In recent years, operational forecasting systems dedicated to subseasonal prediction have been implemented in many NWP centers (including NCEP and ECMWF). Demands for such forecast have been increasing. Types of subseasonal forecast products are, however, still limited. Errors and uncertainties of subseasonal forecasts are still large. With focused research-operation integration, substantial improvement of subseasonal forecast skill and elevated societal benefit are within our grasp.
The weather enterprise has entered a new era of extended weather forecasts beyond two weeks. Science and technology advancements have made it not only possible but also practical to made substantial improvement in extended weather forecasts. What we need is a determination and well thought of plan. A consortium of academic, government, and private sectors within the weather enterprise is recommended to lead the Nation’s effort to make measureable advancement in extended weather forecasts to meet the society’s need and to be the best in the world.
Bill Gail’s testimony provides an interesting perspective from the private sector. Excerpts:
I have talked mostly in terms of weather for the sake of simplicity, but it is important to realize how our strength derives from a breadth of disciplines. For example, we increasingly recognize that space weather is a fundamental counterpart to atmospheric weather. Hydrology and oceanography are key sister disciplines. Disciplines such as coastal meteorology have specific but essential roles.
Climate must be included. For the real world in which my company operates, weather and climate can’t be separated. There just is no good place to draw a line between them. Should we forecast weather out to two weeks, but no longer? Businesses would not like this – forecasts for coming seasons are enormously valuable to companies in energy and agriculture. The travel and leisure industries take an even longer view; they can benefit directly from improved forecasts of the El Niño cycle even years ahead. Construction companies need to anticipate flood zones and coastal erosion decades out. Businesses want to anticipate weather on time scales from months to years, human influence or not. Our commodities markets – from heating oil to orange juice – could not function without seasonal climate forecasts. Whether it is a military strategist analyzing regional vulnerabilities, or simply one of us planning a sunny day for our daughter’s wedding a year ahead, information about climate and its variability is central to the nation’s wellbeing.
Put simply, understanding the fundamentals of climate variability is essential to forecasting weather. What we learn from climate modeling significantly improves our weather forecast skill. Arbitrarily distinguishing between weather and climate makes no sense. Rather than dividing the weather and climate communities, we need to bring them together to improve forecast accuracy at ever-longer timescales.
Cliff Mass has a provocative blog post on the hearing entitled Climate vs weather prediction: do we need to rebalance? Excerpts:
The Democratic side of the committee were clearly unhappy since it is clear that part of the intent of the bill is to rebalance resources in NOAA: more for weather prediction and less for climate.
My take: although this bill has its issues and needs serious revision, I believe that resource allocation has become highly skewed towards climate prediction, to the detriment of BOTH weather prediction and understanding/prediction of climate change. I also believe that a revised bill could be highly bipartisan and a major positive for the U.S. and the world for both weather prediction and climate.
Some supporters of this imbalance argue that climate and weather research are inter-related and thus one should not be concerned about the budgetary issues. That climate and weather research is on a continuum that cannot be divided in any meaningful way. But I would argue that how money is prioritized does make a huge difference and that in actuality the ability to predict the future climate DEPENDS on weather prediction. Let me make the case.
Weather prediction and climate prediction both depend on the same technology: numerical models of the atmosphere. In fact, both the resolution and physics used in climate and weather prediction models are converging rapidly. Weather prediction models are run several times a day and are rigorously verified against a range of OBSERVATIONS. Thus, working on weather prediction allows a cycle of continuous verification and improvement. When you run a climate model into the next century, verification is an obvious problem. And a fact that is often buried is that climate models are often tuned to match the contemporary climate and thus their predictions are suspect.
Bottom line: if you want better climate predictions, you need better numerical models of the atmosphere and weather prediction offers the fastest and most effective route to better models.
There are many that believe that the weather may become more extreme under global warming (although I think there are a LOT of uncertainties in this assumption). But let us assume that extreme weather events (hurricanes, floods, tornadoes, heat waves, etc.) WILL become more extreme and frequent in the future. What is the first thing you need with more extreme weather? BETTER WEATHER FORECASTS. Good weather forecasts have a huge impact in saving lives and property for extreme weather events, something shown for Superstorm Sandy where about 150 died in a region of tens of millions of people (and many who died did so because they ignored the warnings). So if you care about the impact of climate change, you should be an enthusiastic supporter of better weather prediction.
I don’t think I have to work hard to convince you that improved weather prediction saves lives and improves the economy TODAY. With trillions of dollars of U.S. economic activity sensitive to the weather, even small improvements can save our nation tens of billions or more a year. The U.S. has some of the most extreme weather in the world and we require state-of-the-art weather prediction to protect our citizens and to aid economic activity. High-resolution U.S. weather prediction is only done by the U.S. government or U.S. companies/universities–the Europeans will not do this for us. But dozens of major groups around the world are doing state-of-the-art global climate predictions for next century. Quite frankly, if the U.S. wasn’t doing climate simulations there would still be plenty to choose from and lots of research activities using them. And the skill of such climate predictions are uncertain and few nations appear to be willing to make major economic decisions based on them.
The framers of the Weather Forecasting Improvement Act are correct: the U.S. must put more resources into weather prediction research, development, and infrastructure. We have underinvested in weather prediction, with very negative effects on the nation.
The payback on more investment in weather prediction for the nation would be huge. And, it would greatly enhance our ability to predict and warn our nation for any impacts of climate change. If the money existed, it would be nice to greatly increase weather prediction support and leave climate alone. But if resources are tight, the only logical decision is to rebalance our current investments more towards weather prediction.
JC comment: Here is my take: rather than rebalance, we need to integrate. And the focus of the integration should be the sub seasonal (beyond two weeks) to annual timescales. Subseasonal weather/climate models couple an ocean model to an atmospheric model, unlike shorter term weather prediction models that are atmosphere only. The potential economic impact of weather/climate forecasts on this time scale are very substantial. But there are very real advantages for the climate modeling community (long-term century scale) in working through these timescales with a coupled ocean-atmosphere model to improve simulation of coupled modes of variability, including the MJO and ENSO.
Specifically with regards to NOAA, I don’t see why so much of its budget should to to GFDL to support long term climate modeling. The GFDL group is superb, but a major criticism of GFDL is that it has never (since its inception over 60 years ago) collaborated in a meaningful way with the weather prediction branch of NOAA, although that was its originally intended mission.
Getting GFDL to collaborate with NOAA NCEP is low hanging fruit for NOAA in terms of improving its weather forecast models and providing better operational weather forecast services for the nation.
One final comment: While I mostly agree with what Cliff Mass has to say, I would argue that improving the weather prediction models is necessary but not sufficient for improving the atmospheric component of climate models. Certain approximations made, e.g. for moist thermodynamics, are OK in weather models but may cause accumulation of errors for the long integration times of climate models.
They will never get the climate models right until they understand that is snows more when oceans are warm and wet and it snows less when oceans are cold and frozen.
We shud start here and work to improve it. It’s almost as good as the best that money can buy but costs far, far less.
ice has diminished and sea levels have risen because the oceans were cold and frozen and it did not snow enough to replace the ice that melted every summer. That has changed. Now the oceans are warm and wet and the ice will increase because it snows more than enough to replace what melts every summer. Weather Forecasters do mostly know this. They will bring this to the climate table.
JC comment: Here is my take: rather than rebalance, we need to integrate. YES!
Radiation, Water, Ice and More
Radiation cooling has bounded the temperature of the Earth Forever. It works, but the bounds are very large by modern standards. It has no set point.
Only Water changes state in our comfort zone with a set point. The Polar Ice Cycle developed and tightened the bounds of Earth Temperature.
Water Vapor accounts for ninety some percent of the Radiation Cooling.
Water was put on earth to regulate temperature and help living things. Water, Water Vapor, Ice, and Clouds, that is: Water, in all of its states, does control the temperature of earth.
CO2 was put on earth to help a trace amount with the cooling and to help a huge amount with the living things.
The drift of the continents and the evolution of the ocean currents and the development of the polar ice cycle perfected the bounding of earth temperature.
ICE and Water has a set point. When oceans get warm, sea ice melts and Huge Snowfalls Occur. This builds ice volume on land which advances after some many years of snowfall. When the Polar water gets cold and frozen the huge snowfalls stop. The ice volume is like a huge charged capacitor and the ice advance continues, but without snowfall, ice volume starts to decrease immediately. More ice melts every summer than gets replaced. The ice advance continues and runs out of capacity after a good number of years. Clouds are used in this process. When the ice is building it is protected by many clouds. When the ice is being removed, the clouds are not there.
The experts think the ice volume increases right up to the ice extent max, but it really cannot happen that way. They build ice because something external caused earth to get cold. They take away ice because something external caused earth to get hot. They start taking away the ice when earth is still cold. They don’t have an external forcing that has a set point. Their basic theory cannot work.
We need the Cycles. We do not have one fixed temperature that can be maintained. We have a cycle with powerful bounds. Ice and Water has a set point and does maintain the bounds.
Radiation can bound temperature in wide bounds with no set point and the bounds can change a lot with changes in external forcing. This is how earth was. Radiation Bounds have no Fixed Set Point.
Ice and Water have a fixed set point and the bounds stay the same with large changes in external forcing. This is where we have been for the 800k years of large cycles of ice age and warming. The modern ten thousand year cycle is even more tightly bounded. Earth evolved to this state and it will continue. This is how Earth is now:
The Sun melts Ice every Summer. The Sun melts Land Ice and Sea Ice every Summer. Snow falls and replaces Land Ice and the cold freezes Sea Ice every Winter. When there is more water exposed in the Summer, it Snows more in the Winter. When there is less water exposed in the Summer, it Snows less in the Winter. This is the SET POINT that puts the tight bounds on Temperature.
A trace of CO2 cannot kick us out of this modern paradise; it can only help green things grow better using less water.
Start in a warming phase, such as the past 130 years, since we had thermometers and could measure and record temperature. The land ice has been receding and the oceans have been warming and rising and sea ice has been receding. With the warming of the polar oceans, the snowfall has been increasing. The snow falls on bare ground and on glaciers and ice fields. The snow that falls on bare ground at the edges of ice fields and around the tails of glaciers mostly melts every summer. Much of the snow that falls on glaciers and ice fields becomes multi-year ice. This multi-year ice builds and gains weight and after some years the multi-year ice starts to advance. As the multi-year ice advances, earth cools and the oceans cool. Ice volume is still increasing. At some point the oceans get cold enough and the water freezes and the snowfall stops or greatly reduces and the ice volume stops increasing and starts decreasing. The Piled up ice is still heavy and continues to advance. The Piled up ice gradually runs out of capacity to continue pushing the ice and the advance stops. The cooling stops. The sun has been melting more ice than was replaced, but now the ice starts to recede and the earth starts to warm again. Land ice is receding and the edges of the sea ice start to recede. At this point you can go start at the beginning of his paragraph again, and again.
Consensus Theory uses orbit parameters and solar cycles and CO2 to make earth colder and then builds ice by letting the snow that fell on bare ground survive the summers and grow the ice at the tails of glaciers and edges of ice fields. There is no evidence that supports this method. The glaciers advance and drop stuff they picked up on the way. They do not develop at the edges and tails. They develop at the tops and then when they are big enough they advance.
So tell me HAP, let’s say this theory of yours were to gain consensus agreement, but then ends up being flawed and bad policy comes out of it harming untold numbers of people. What kind of regulations should be passed so folks like you don’t cause such harm again?
The control of the atmosphere by the biosphere is a huge missing analytic element in all this. Consider: free oxygen in a planetary atmosphere cannot endure without constant refreshment and replacement by organisms (plants). All of the O2 in the air (20%) was once therefore CO2, before plants stripped the C for their use. Hence CO2 was once >= 20% of the atmosphere, not the current 0.03%. Life pushes the CO2 level down till it is too expensive to extract, then it wobbles around that point. The end product is geology: limestone, etc.
If mankind has a Gaia-given role in this scheme, it is to return as much of the sequestered C to circulation as possible. Carbon taxes should hence be negative (subsidies), not positive (costs), for this reason.
HAP…I’m sorry, but I find your analysis of Earth’s warming/cooling to be without substantive support. IF you want your posts to have some credibility, I suggest that you include references, especially, peer-reviewed journals or acclaimed texts. Thank you.
Hi Judy – I found your post including Cliff Mass’s comments insightful. What we need (and this covers your conern) is to focus on “predictability” on different time scales, not on blindly making multi-decadal climate predictions. I discussed this in my posts, for two examples, see
JC is right too on integration. Surely the blindingly obvious way forward is integrated research combined with the only the level of forecasting that has demonstrated reliability.
This link below is relevant to the state of climate prediction. It shows that for longer time scales there is still a long way to go in predicting decadal swings. It definitely comes down to the ocean, where there is a big challenge with initial data and resolution to get the variability right.
By the way, GFDL are involved with weather forecasting, but primarily just hurricane forecasting.
JC, I agree with your comments with some reservations. The devil is in the details of the plan. If integration means that model development and tuning is the main thrust and observational studies are diminished, I would remain as disappointed in the approach as I am today. If in another five years we learn little about clouds, aerosols, ocean attributes, the sun, and other natural variations then it is just more scarce resource wasted.
Keep up the good work!
Good Heavens, this is like trying to run your democracy on popular principles. Can climate understanding come from improving weather models? Perish the thought.
The good news this is being actvely discussed, perhaps as a follow on to the gross past imbalances in funding improved computing power between climate ( too much) and weather (too little). Quarterly forecasts would require integration, something the MET is trying to do but so far has failed miserably at. Improvement would seem to require the sorts of advances in both data quality (UHI, properly maintained stations) and physical theory and model representations thereof (ocean cycles, clouds,…) that climate seems to lack.
There are many problems in climate that are not helped by having a better weather model, especially as you get to longer terms. The whole issue of feedbacks and the long-term energy balance don’t show up in weather forecasting. Also aerosol and volcano effects on the forcing, rising CO2 and methane, melting sea-ice and glaciers, long-term vegetation and land-cover changes, ocean heat transports and circulation changes, are crucial things for climate models, but are not needed for accurate weather forecasts out to two weeks. So I don’t believe there is such an overlap as Cliff Mass implies. Improving weather forecast models should be a priority for its own usage, but I don’t think the biggest climate model scientific uncertainties are in the atmospheric model itself. Improving the sub-seasonal to seasonal forecasts would benefit climate models, because that probably will demonstrate what is needed to represent the ocean (and sea ice) well enough to predict it beyond one ENSO cycle, which is a tough target today.
The fast feedbacks (particularly clouds) operate at the shorter time scales; this is the feedback with the greatest uncertainty. The biggest problem with climate models is their failure to correctly simulation the teleconnection patterns of the coupled ocean atmosphere system. Getting the MJO correct would seem to be a prereq for getting ENSO correct, which is a prereq for getting the PDO correct, etc. Looking at coupled atmosphere-ocean models on these shorter timescales would help deal with the biggest problems currently facing climate models.
On the other hand, getting global cloudiness right is not something that is a priority in weather models, as much of it is over the oceans where people don’t care so much about it, but yes attention is needed to this if the oceans are coupled. I agree on the other points.
Most of the feedbacks eventually fizzle out approaching asymptote set by the natural variability. Certain regions (North Atlantic, North. and Central Pacific) are drivers of such variability, while polar regions (Circumpolar Current and to some extent Beaufort Gyre) play important stabilising role.
‘”The Madden-Julian Oscillation is the bridge between weather and climate”. – Chidong Zhang, Chief Scientist
– http://www.youtube.com/watch?v=bA230B4w9t8 –
‘Scientists need to better understand the MJO, both to improve long-range weather forecasts and seasonal outlooks worldwide and to perhaps make the leap to longer-term forecasts of climate that may extend years into the future.
That’s the impetus behind DYNAMO (Dynamics of the Madden-Julian Oscillation) a six-month field campaign to help improve those long-range forecasts and enable scientists to further refine computer models of global climate. The National Center for Atmospheric Research is providing major observing tools to the science team and helping to oversee operations and data management for the project.’
‘Hypothesis I: Deep convection can be organized into an MJO convective envelope only when the moist layer has become sufficiently deep over a region of the MJO scale; the pace at which this moistening occurs determines the duration of the pre-onset stage.
Hypothesis II: Specific convective populations at different stages are essential for MJO initiation.
Hypothesis III: The barrier-layer, wind- and shear-driven mixing, shallow thermocline, and mixing-layer entrainment all play essential roles in MJO initiation in the Indian Ocean by controlling the upper-ocean heat content and SST, and thereby surface flux feedback.’
Although I think the initiation of both ENSO and the PDO has more to do with solar UV modulation of the SAM and NAM pushing more or less cold water into the Peruvian and Californian currents respectively. The MJO may be involved in the initiation of the discharge of the western Pacific warm pool and the evolution of El Nino.
“The whole issue of feedbacks and the long-term energy balance don’t show up in weather forecasting” – Jim D
“Looking at coupled atmosphere-ocean models on these shorter timescales would help deal with the biggest problems currently facing climate models.” – curryja
Might there be a way of studying the ways in which weather models go off track and need to be re-initialized which could throw some light on the longer term forcings and feedbacks which the weather models are not taking into account? Also, as I understand it, weather models are applied to local and regional predictions which, again as I understand it, climate models are not very good at. Might not there be a way of studying the output of weather models to better understand the ways in which the climate models fail at regional scales?
I think the main reason for weather models going off track is because of the growth of small local errors to large ones (Lorenz butterfly effect), so their emphasis is getting the initial state right. Climate drifts show up as biases but these are not harmful to weather forecasts in the short range they are used for, because they grow slowly and don’t affect the patterns which are all-important. The focus of weather on the patterns and climate on the mean makes them rather orthogonal in their priorities.
Regionally too, the weather models don’t run long enough to go far off track, which climate models may after months to years of running without re-initializing. This is an area where seasonal prediction can help climate models, but so far that is not a solved problem.
‘Small local errors’ are crucially important, after all, most of us work and live within our local microclimate.
If you plan that it will rain overnight for six hours but instead it rains during the day for the same length of time that can have a dramatic effect on peoples days, especially those in farming or in tourism
Exactly, this is a weather priority, not at all a climate one. They measure success by getting this type of thing more exact, or at least giving a decent probability for hours to days ahead.
On the recent “IPCC Discussion” thread, I posted the following quote and comment:
“’For most users climate and weather are mutually interchangeable. It is, therefore, imperative for climate and weather services to operate in close tandem, so as to be seamless to the end-user.’
Translation: we have blown our credibility, but everyone acknowledges the importance of short term weather information. So we want to clothe our wolfish drive for decabonization of the global economy in the sheep’s clothing of weather services.”
I think it fits here too. What we are seeing is the results of poll testing and focus group efforts to “reframe” the climate alarmist industry because of the failure of models to come even close to reality, and the lack of catastrophic warming the last x number of years. This is both organizational and advertising reframing.
It’s like this scene from the classic Blazing Saddles:
Reported temperatures aren’t rising, catastrophes aren’t coming fast and furious, people don’t believe the thermal sky is falling…We have to save our budgets gentleman! We have to latch onto “weather” and redefine it to include climate, and all the models and research we have been funding for decades.
‘Here is my take: rather than rebalance, we need to integrate. And the focus of the integration should be the sub seasonal (beyond two weeks) to annual timescales. Subseasonal weather/climate models couple an ocean model to an atmospheric model, unlike shorter term weather prediction models that are atmosphere only. The potential economic impact of weather/climate forecasts on this time scale are very substantial.’
Coming from a tourist area that is geographically very close to the Met office we are acutely aware down here of the importance of giving accurate weather forecasts.
One of the large resorts along here even set up their own weather service following a string of poor forecast by the Met office that didn’t come true.
We have the Met office app on our phone. When the weather is settled, that is to say a long period of either overcast or sunny weather, they are usually accurate. When the weather is mobile we joke that the app changes its forecast 5 times a day and even then can be wildly wrong.
Perhaps its different in the States, but a three day forecast in settled conditions is the most we can hope for. A month No. A year? You have to be kidding.
I KNOW there is supposed to be a huge difference between forecasting tomorrows weather and the climate in 200 years time, but as they don’t even seem to know the UK’s past climate over the past two hundred years I have no faith at all they can model it for 200 years hence, or a year hence, or a month hence, or two weeks hence…3 days perhaps, if its settled.
Mind you my piece of seaweed and £50 weather station can do that.
What was it Nassim Taleb said about prediction
in ‘The Black Swan?’ Remind me. Lol.
William Gail, “an interesting perspective from the private sector?”
William Gail is apparently a true believing acolyte of the Church of CAGW.
“Whether we start today or in a decade, it is inevitable that we will begin to apply our newfound capabilities to actively manage–even engineer–climate.”
One wonders how such a true believing warmist is “President-Elect, American Meteorological Society.”
The answer is, progressives are attracted to positions that they think have more “impact,” or “influence,” aka power. That is why the leadership of so many organizations are much further left than the members they represent. Progressives work hard to become editors, department chairs, leaders of professional organizations, so that they can then use the perceived authority of their profession to advance their agenda.
It is also a common tactic of the more, shall we say well informed, progressives to try to disable opposition from within. It is not a conspiracy, it is a movement. Meteorologists are by and large skeptical of the CAGW movement. How do they elect a warmist to head their organization? Because they are busy doing their jobs, and not obsessing about who “controls the message.”
Now a full throated apologist of all things CAGW is cited as both a voice from the private sector, and “President-Elect, American Meteorological Society.” Gee, that must mean he has more credibility than, say, someone like a climate scientist of government employee who derives their income from the government. Right?
Uhhh, not so much. Who are this innocent capitalist’s customers?
“The company has made strides in the wind industry by providing wind forecasting services for Minnesota-based Xcel Energy’s wind turbine fields.”
“3. Other than the wind industry, what other sectors are you involved?
We do the road weather forecasts for departments of transportation, a good example being the Colorado Department of Transportation. We provide the pavement forecasts for the major roadways throughout the state.
… The other thing that we’re starting to do is we provide global weather forecast for media companies that don’t necessarily do their own forecasts.”
Wind farms and governments and media, oh my!
What a shock. A prototypical crony capitalist favors massive increases in the funding of climate research, and has jumped on the reframing bandwagon of merging weather and climate.
I don’t want anyone managing our climate when they clearly don’t even understand what is going to happen next year or even later in this year.
If you think we might not do it if we wait, that is all the more reason to not do it. William Gail is suggesting we do something before people decide it would be the wrong thing to do something. Wait and study the data. We are getting more better data. Study the data and learn more before doing something that turns out to be really bad. We worry about nature harming people. Now you want to take in into our own hands to harm people.
What we get consensus on turns out to be wrong more often than right.
I just read about salt. How long has this consensus been wrong.
Couple of comments:
IMO the real difference between weather and climate as objects of study is often missed, including here. If we regard the current state of the atmosphere/ocean etc. as a point in an n-dimensional space (with very large n!), its evolution represents motion within a “basin of attraction” towards or along an attractor. Climate should more properly be regarded as the actual “geography” of the basin(s) of attraction itself. Changing a boundary condition, such as pCO2 or solar “constant”, changes the configuration of that “basin of attraction”, with resulting implications to both the average location of the state in n-space, as well as the PDF regarding its evolution. Weather, OTOH, is the actual state and its actual evolution.
IMO the “brute force” approach to climate modeling is ultimately doomed: no matter how much more powerful and numerous the computers applied (over the next few decades anyway), the actual behavior of the atmosphere/ocean will remain out of reach; it depends on individual agents at a millimeter scale, if not smaller. There needs to be a fundamental and radical change of approach to the objects actually being modeled.
I wish I could be more specific what the new system needs (IMO) to be like, but I’m still struggling with how to communicate it. One very vague analogy might be in studying turbulence: modeling individual points in the fluid vs. studying the stochastic evolution of vortices and so on. The behavior of evolving vortices might be described (statistically) and parametrized, so that their probable evolution can be described in a (relatively) small amount of information without the need to actually model the individual points in the fluid.
Perhaps something analogous could be done for the atmosphere and ocean: defining specific agents such as the jet stream, waves in the jet stream, tropical low-pressure zones, etc. as specific objects, whose behavior can be statistically defined in terms of other nearby and teleconnected objects. Other objects would be static, the result of geographic features and their interaction with the atmosphere/ocean.
In the search for such objects, it might be possible to make use of neural networks, which have the ability to identify patterns in apparently chaotic behavior without needing to be programmed with how those patterns work. Once such patterns are found, both the real-world behavior and the successful neural network recognition/simulation of such patterns could be studied to attempt to define how they work. Much more detailed use of the traditional modeling techniques would then be able to play a big part in both studying, and correctly parametrizing, the behavior of the objects generating these patterns.
“I wish I could be more specific what the new system needs (IMO) to be like, but I’m still struggling with how to communicate it.”
What would a system or model have to be like to predict where the next bolt of lightning will strike? To predict the next roll of a die? To predict the shape of the next cloud over my house?
Some things can’t be modeled to produce any accurate predictions. Climate is probably one of them. Certainly it can’t be right now.
Interesting. There have been some efforts using object oriented predictions for short term weather features but this is something that has been largely unexplored in atmospheric sciences. Here is the one study i am aware of, by Elmar Reiter
Thank you for the link, Prof. Curry. First thing I noticed:
So I wasn’t surprised the report date was 1992.
I’m going to study it for a while, maybe get back with some thoughts later. AFAIK chaos theory has come a long way since then, as has object oriented programming.
Agreed, I think this idea is worth pursuing
Koutsoyiannis et al. are developing stochastic models to better incorporate climate persistence (Hurst Kolmogorov Dynamics). These may provide more accurate probabilistic predictions than current brute force GCMs with their biased climate sensitivity. e.g. see
Koutsoyiannis, D., Encolpion of stochastics: Fundamentals of stochastic processes, 30 pages, Department of Water Resources and Environmental Engineering – National Technical University of Athens, Athens, 2013.
Mystegniotis, A., V. Vasilaki, I. Pappa, S. Curceac, D. Saltouridou, N. Efthimiou, G. Papatsoutsos, S.M. Papalexiou, and D. Koutsoyiannis, Clustering of extreme events in typical stochastic models, European Geosciences Union General Assembly 2013, Geophysical Research Abstracts, Vol. 15, Vienna, EGU2013-4599, European Geosciences Union, 2013.
Focusing more on weather/nearer term forecasting combined with stochastic models would help restore the scientific method with rapid feedback of how the forecasts compare against consequent evidence. Such improvements to accuracy will provide greater confidence in the longer term forecasts/predictions. Current climate “projections” have “gone off the rails” and cannot be relied on until serious evidence based feedback correction is implemented. This bill should help prompt this major needed strategic correction.
People have very little appreciation of how Herculean the task of being able to execute software written a hundred years ago.
Oh wait. That’s not internet time. Make that software written a thousand years ago. ;-)
I’m here to tell you, as a Wintel R&D engineer all my adult life, Windows 3.x sucked the big one. As a user I went from MS-DOS to Windows 95. The only time I ever looked at Windoze before that was to make sure my hardware and/or firmware worked with it. When Windows 95 came along I was hooked from the earliest beta versions and considering I was laptop BIOS programmer at Dell at the time I got the latest builds from Microsoft as early as anyone. Association between big box makers and Wintel was very close and we were closer than most especially with Intel.
I didn’t have that luxury. I was working in (IT support for) the loan originations department of a major bank at the time, and we had business users who had experience with 3.0, 3.1, and 3.11 at home and installed them on department machines at work, then called IT when it broke something in our DOS/Novell-based systems. We ended up having to support Windoz on a DOS/Novell origination system whose documentation specifically said it didn’t support running in a DOS window on 3.11 or earlier.
I’m not sure which was its bigger problem – no preemptive multitasking or trying to get 10 pounds of graphics processing out of 1 pound graphics adapter.
Novell Netware brings back a lot of memories. I was doing a lot with single board PC compatible diskless workstations circa 1990. Earlier than that was a lot of Arcnet but Ethernet came to rule of course. Ever heard of Turbo-DOS, Concurrent-DOS, or even Concurrent CP/M? I done ’em all.
I’m currently having a love affair with Node.js that’s going on two years old.
Hello Prof. Curry. I’m still digging into it, but I discovered this: An Object Oriented Shared Data Model for GIS and Distributed Hydrologic Models by Mukesh Kumar, Gopal Bhatt and Christopher J. Duffy International Journal of Geographical Information Science Vol. 24, No. 7, July 2010, 1061–1079
I’m not sure how much relevance it has yet,
There are a number of technical issues involved in using OOP for creating models, the most important (AFAIK at this time) being dynamic memory allocation and access to object methods and data in multiprocessing systems. Still looking into where the current modelling is in this regard.
My immediate approach would be to bite the bullet and create the infrastructure for queued asynchronous access to object methods (by data rather than reference), allowing model coders to reference object methods without worrying about what system(s) they’re running on.
I’d also favor writing the new systems in Java, with its automated garbage collection, as well as its theoretical ability to run compiled code on any platform. One likely advantage to object oriented modeling (OOM) is that it would probably scale well down to sizes suitable for local now-casting and short-term forecasting, This means that the same code should ideally be able to run on local workstations with standard JVM’s.
I know there are performance issues with Java, but that could probably be addressed at the level of JVM development. That would also be the place to address issues regarding multiple processors and distributed systems, where it would be transparent to coders.
I could see an automated version of Reiter’s thunderstorm efforts as constituting “low-hanging fruit” from a ROI perspective: It could potentially offer better forecasts and warnings for tornadoes. OTOH, if I were going to try to develop prototype methods and protocols for OOM, I’d probably want to start with simple cumulus convection cells.
Thunderstorm and Deep Convection Warning Project
It is often assumed that doubling CO2 is going to make the climate too alien or challenging for current models when it is actually only a 1% perturbation of the current climate. This is why weather and climate models can still apply for future climate and tell us what it will do to good accuracy. It is not going to be like Venus. The perturbation is less than the daily or seasonal change or latitude variation that they already handle well.
Models are tuned of course – but solutions diverge into the future.
From Slingo and Palmer – divergence of a perturbed model is shown.
All models diverge within the range of feasible inputs and any single solution is merely a partial – and quite uncertain and qualitative – sampling of the solution space of possible climate states. Any particular solution is chosen on the basis of expectations about where the true solution lies. it is subjective rather than emerging objectively from a unique calculation. You might assume that solutions in ensembles of opportunity cluster around the perfect solution – so the mean is more likely to be accurate. But that is an article of faith rather than a scientific precept.
The perturbations in energy flux at TOA are not merely daily, seasonally or latitudinally large – but vary on long timescales as a result of variability in ocean and atmospheric circulation. We can see that in the satellite records – and infer it from the long term temperature records millennially to millions of years. Climate changes at all scales as sub-systems interact nonlinearly – but depending on models to predict it is a crapshoot.
There isn’t really any need to predict where the next bolt of lightning would strike, if you can produce a good tight PDF relative to the location and configuration of the thunderstorm involved. And if there’s a lightning rod, it’s a reliable prediction that lightning is much less likely strike near it.
And your house is too small, it is possible to make some statistical predictions regarding larger features.
Whenever there is a proposed or real organizational “restructuring” , resources are re-allocated. Most times the facility gets a make-over, and in this case of rebalancing, the computer hardware/software is re-focused. There may be new hardware added or others discarded.
Where the rubber meets the road is in the personnel changes: who gets to stay; who has to leave; are there resources to squeeze in someone from the outside.
My experience has been that government has a very hard time with restructuring. Nobody wants to leave and various department heads look for ways to reshuffle the same personnel into other newly created boxes. Having climbed the governmental power ladders, managers have developed political ties that bind so ever tightly, worse than the Gordian Knot.
The problem of keeping the same people is that you keep the same ideas. As many times with the growth of government, agencies linger on, even re-titled, still doing the same thing, the same way as before. Because the re-titiled agency doesn’t get the job done, a brand new agency is created with personnel added; government growth.
There are several ways to cut the Gordian Knot. Overwork the bureaucrats, give them more to do with less time; make work projects. Some people will leave. Redefine the work and credentials such that nobody in the existing agency has the proper credentials: narrow focus to credentials. Cut internal lines of agency horizontal communication: everyone has to report vertically in a narrow organizational structure; tribal like.Temporarily close sections of the agency without expanding; people laid off for a year or two or until their credentials no longer apply. All of the above is to reduce the workforce to meet a restructured budget.
Once the old agency is but a shadow of its former self, now you are ready to tackle a new project with fresh ideas and fresh personnel, like understanding the weather in subseasonal increments.
Climate scientists can go on lecture tours, teach, consult and project how long their retirement moneys will last via Monte Carlo simulations.
Actually, the people working on developing climate models are for the most part capable of working on weather/seasonal climate models. In fact much of the ‘talent’ in the modeling field is working on the climate rather than weather models, so this would not be particularly disruptive in terms of personnel.
On the other hand, climate people don’t care so much about getting that tornado or hurricane prediction right which relies on getting good initial data into the forecast. They have a more big-picture view about getting the mean diurnal cycle and global energy balance right. They are in different worlds, orthogonal, as I said above, and it takes a lot of adjustment in either direction.
The way they address diurnal cycle and global energy balance is largely through working on the cloud parameterizations; climate modelers could learn much but considering the evolution of clouds in context of weather system.
Yeah, their love of ensembles kinds of gives it away. In other words a punt is a punt whether it’s weather or climate.
I certainly acknowledge your well-informed knowledge of the many capable people who are currently doing climate research who could engage in sub seasonal to annual weather forecasting. Most of what I had in mind regarding changing paradigms was the leadership roadblocks to refocusing; as for instance by current members of the Hockey Team.
Every time a reporter sticks a microphone into the face of a member of the Hockey Team, the Science news section of the NYT has their utterances. Their entire self identity is wrapped up in the demonizing of CO2. Pretty hard to give that up the rush to see one’s name on Broadway (news stands).
Like an locally invading malignancy, reaching into the good to serve the bad, the Hockey Team’s reach is extensive. The best chance to alter the course of events is wide field surgical excision.
IMO, the fastest way to change NOAA’s GFDL exclusive focus from climate to weather is first to shrink the agency, then rebuild with people familiar with working in interdisciplinary settings. Not every capable scientist is able to work collaboratively. To address weather forecasting, I believe emphasis is on respecting what each brings to the table.
I don’t think any members of the ‘hockey team’ are employed by NOAA.
Judith Curry, the closest there is in the NOAA is Tim Osborn. I don’t know if he counts as a member of the “Team” or not. I’m not sure just who is and is not part of the team anymore.
Keith Dixon, a modeler at GFDL; “The climate is warming, and we can say why.”
Wasn’t he a member of the Team?
Climate scientists can go on lecture tours, teach, consult
They clearly don’t have the Theory right yet.
We Do NOT want them to lecture or teach or consult. They have already done too much damage brainwashing our children and grandchildren. I say we pay them to be quiet. That would be our best investment.
Hmmm, censorship by paying hush money…. Clearly HAP you have taken a wrong turn.
Needs must, when the Devil drives.
Looking at the return we get on our tax dollar, it is a no brainer. One can rightfully argue that increasing our understanding of climate is a worthwhile goal. However when dealing with finite resources investing in tools which will improve short to medium term weather forecasting clearly offers a far greater return than what we have seen to date from GCMs
Hi Judy – You write
“The fast feedbacks (particularly clouds) operate at the shorter time scales; this is the feedback with the greatest uncertainty.”
I suggest we also do not know much about the slower nonlinear feedbacks; e. g. see Shaun Lovejoy’s article “What Is Climate?” http://onlinelibrary.wiley.com/doi/10.1002/2013EO010001/pdf
“At timescales around 10 to 30 years these weaker and weake fluctuations—whose origin is in weather dynamics—become dominated by fluctuations from increasingly strong lower frequency processes. These fluctuations are due not only to changing external solar, volcanic, orbital, and anthropogenic forcings but presumably also to new and increasingly strong slow (internal) climate processes, such as deep-ocean or land-ice
dynamics or a combination of the two: forcings with internal feedbacks. The result is a climate regime with fluctuations growing with timescale in a weather-like manner.”
Such slower feedbacks appear to also be highly uncertain.
Hi Roger, I agree that the deep ocean feedbacks are at the heart of climate model uncertainties; my argument is that we won’t make progress in sorting this out until we have better understanding of the shorter term teleconnection processes involving the coupled ocean/atmosphere system, which a focus on subseasonal to annual scales would help improve our understanding and modelling capabilities on the longer time scales.
deep ocean feedbacks are at the heart of climate model uncertainties; my argument is that we won’t make progress in sorting this out until we have better understanding of the shorter term teleconnection processes involving the coupled ocean/atmosphere system
As atmospheric teleconnections are simplistically speaking a redistribution of mass, the ocean processes are similar only bigger.The schism being between thermodynamics and dynamic systems theory,the later being where the systems tend to be either in mechanical equilibrium (or moving towards) without being in thermodynamic equilibrium.
Natural fluctuations in CO2 fluxes and their consequences also appear to be strongly underestimated. e.g., see Murry Salby’s findings.
Natural fluctuations in CO2 fluxes are perfectly understood and are perfectly seasonal, leaving the underlying trend and perturbations due completely to man-made fossil fuel emissions.
Scroll to the bottom where you will see the regression fit
Throw more money at it. Yeah, that’s it. That’s the ticket. That’s always works. We have cures for the common cold and cancer and AIDS because we increased research spending.
Could climate models, with a scaled-down grid, be used for weather prediction? If not, why?
This is already done at some modeling centers. UK uses the same atmospheric model for both weather and climate. ECHAM climate model uses a version of the ECMWF weather model. Again, the rationale here is to use a an atmospheric model that has been extensively evaluated against weather data. Necessary but not sufficient for a good climate model. Note: the US does not do this.
UK weather predictions are often source of national mockery. About a month ago they announced ’10 years of miserable summers’; for the last two to three weeks we had the best summer weather for years, if it continues for two or more weeks it may be compare to the one of 1976.
My recollection of 1976 in the UK, for those who don’t grasp the reference, is that there was no cloud from June to October, just long, sunny days and much higher than average day-time temperatures.
I was in UK then too. It was great. Ian Chappel, Captain of the Australian cricket team, and we beat the poms on their home turf!! :)
One example of the never ending entertainment was the discussions on the Tube (London Underground). Aussies stood at each end of the carriage and discussed the cricket with the Aussies at the other end of the carriage while the pommies, in their suits and bowler hats, tried to focus on reading the morning paper without looking up. Great sport!
I remember one pomme telling me at the time, you can’t get into Earl’s Court without an Aussie passport. Which of course, is how it should be!
Around that time, Rupert Murdoch bought ‘The Times’. That was great sport too. We asked the pommes if the BBC was for sale too.
Of course, I had no part in any of this, simply an innocent bystander. :)
Actually, that was 1975, and also a great summer.
Are these the same models that predict exactly where
hurricane is going to make landfall? I hope not.
‘The global coupled atmosphere–ocean–land–cryosphere system exhibits a wide range of physical and dynamical phenomena with associated physical, biological, and chemical feedbacks that collectively result in a continuum of temporal and spatial variability. The traditional boundaries between weather and climate are, therefore, somewhat artificial.’ http://journals.ametsoc.org/doi/pdf/10.1175/2009BAMS2752.1
Hurrell et al were of course arguing for a few thousand times more computing power for ‘seamless prediction’ of weather and climate. This is one way of proceeding – but with little confidence in the numerical representation of the physics of the coupled nonlinear system.
Frankly – for seasonal and longer outlooks the simpler methods of correlation of rainfall, temperature, cyclone frequency, etc. with patterns of ocean and atmospheric – especially sea surface temperature – seem more promising. Here for instance is a drought index for the US.
You can see for yourself the connection between US drought and important patterns of SST.
Here’s one for the next 3 months for Australia.
A negative IOD is the primary driver for predicted winter/spring rainfall. Cooling in the eastern Pacific has implications for the Austral monsoon and cyclone frequency and intensity in the western Pacific and the Atlantic.
Here’s one from Roger Pielke Jnr and Christopher Landsea – http://www.aoml.noaa.gov/hrd/Landsea/lanina/
In a cool Pacific Decadal Variation mode – such as now – the expectation is for more frequent and intense La Nina.
ENSO has been studied intensively for many years. It remains the case that numerical prediction is no more reliable than tossing a coin after 3 months and at specific times of the year. The rest of these standing patterns of ocean and atmospheric circulation are yet more enigmatic. The teleconnections in the global coupled system are just beginning to be investigated.
‘Climate is ultimately complex. Complexity begs for reductionism. With reductionism, a puzzle is studied by way of its pieces. While this approach illuminates the climate system’s components, climate’s full picture remains elusive. Understanding the pieces does not ensure understanding the collection of pieces.’
There is no argument that numerical models of climate should be neglected entirely – simply that the limitations that will remain for the next however many decades should be recognized. Other methods exist using the traditional techniques of correlation and the powerful pattern recognition capabilities of the human brain that may be more useful for some time yet.
Chief – I looked at your reference at photobucket.com. Very colorful, but insufficient data. What do your colors mean? What do the maps mean? Please include a complete reference to McCabe (2004). I am not disputing you, I just don’t understand it.
‘Although long considered implausible, there is growing promise for probabilistic climatic forecasts one or two decades into the future based on quasiperiodic variations in sea surface temperatures (SSTs), salinities, and dynamic ocean topographies. Such long-term forecasts could help water managers plan for persistent drought across the conterminous United States (1). The urgency for such planning became evident when much of the U.S. was gripped by drought in 1996 and again in 1999–2003, evoking images of the dry 1930s and 1950s. Analyses and forecasting of U.S. precipitation have focused primarily on the Pacific Ocean, and more specifically on oceanic indices such as those used to track the El Nino Southern
Oscillation (ENSO) and the Pacific Decadal Oscillation (PDO). Much of the long-term predictability in North American climate, however, may actually reside in both observed and modeled multidecadal (50–80 years) variations in North Atlantic SSTs (2–7).’
A 2007 reference – http://wwwpaztcn.wr.usgs.gov/julio_pdf/McCabe_ea.pdf
The new climate change catastrophe scenario?
– http://www.youtube.com/watch?v=iwsqFR5bh6Q –
Be very scared.
Thanks for clarifying. We are in agreement. As you write
“….we won’t make progress in sorting this out until we have better understanding of the shorter term teleconnection processes involving the coupled ocean/atmosphere system, which a focus on subseasonal to annual scales would help improve our understanding and modelling capabilities on the longer time scales.”
This certainly is a necessary condition for any hope on longer time scales.
Call Bob Tisdale and then let’s figger out how the sun drives the coupled ocean/atmosphere system, or if.
It is a shame that weather forecasters in the US have to use the crumbs on the plate of the European models (ECMWF) for accurate forecasts, because the better US computer power is focused on climate.
We need to reallocate some of the climate horsepower to weather forecasting, whether that helps climate modeling or not. This is especially true in the 1-2 week frame, where ensembles are required and thus more computing resources are utilized. But even the short term is getting shorted, so to speak. More horsepower would allow finer resolution models, and allow them to be run more often.
And… even this is looking at things with too narrow a focus. Hazard mitigation – especially from tornadoes and related severe events – could be improved with not only better models, but with money allocated to better instrumentation. For example, the atmosphere’s vertical profile is, for the most part, only sampled twice a day at 400km horizontal resolution, which is absurd (yes, there are other inputs – radiometers and ACARS). NEXRAD radars cause a very high false positive rate (false tornado warnings) as a result of their sparseness and slow scan speed, and this high positive rate results in more death and injury as people ignore warnings (per study of Joplin survivors, and general observations of tornado alley behavior by many observers). So let’s redirect some money into instrumentation, too.
The vast expenditure in climate modeling has produced little benefit other than careers for climatologists and a slowly improving, but still very weak understanding of climate. The bang for the buck just isn’t there – the spending is way out of proportion to the benefit. Pure science always has merit, but climate modeling has gotten disproportionately vastly greater funding, not for its scientific merit, but because of the dishonest politicization of the issue of global warming, errrr, climate change.
We need to recognize that weather forecasting and shorter term climate work produces measurable near-term benefits, and should thus receive higher priorities.
That’s what I am interested in right now, the vertical profile and how it changes due to GHGs
Interesting stuff on geopotential height in the top post.
“Weather prediction and climate prediction both depend on the same technology: numerical models of the atmosphere. “- Mass.
Yes and no. Because of the different tine scales, atmospheric physics is more important to weather forecasting than ocean physics, but for climate forecasting ocean physics is essential. owiing to the long heat transport delays in the oceans.
Yes, weather ,tenperature, is really the time derivative of climate, so there is a continuum which should be exploited when useful. But none of the climate models correctly forecast the present pause. In fact they seem to be blind to the on/off nature of global warming, although the present ‘pause’ is the second coming of that event, The first was in 1940 to 1970 when global temperature actually fell.
The integration of meteorology and climatology seems the way to go but weather forecasts at the local level seems spatially too low in resolution to be effective proxies for climate prediction purposes.
What seems needed is a greater emphasis on regional trends at the sub-seasonal level because this would have benefits for communities far exceeding the value of the current group of global climate models.
weather is climate – GLOBAL warming is theology
Pingback: Climate vs. weather prediction? Really? | Living on the Real World
Our forecasting here in rural Oz is now very peculiar. Predicted conditions – wind, rain, temp, percentage of probablity etc – seem independent of one another, as if they were done by different people on different systems. As the predicted day approaches, the previous forecasts change in an apparent effort to match what is actually happening…but it is usually only the report of yesterday’s weather which is accurate. It’s not that the predictions are wrong, just that they are handy but blurry speculations and should be expressed as such, preferably by people who get out a bit.
Some observers have a nose for weather, as well as some knowledge. They can be good to consult. Radar and sat are very handy. Strict building and development standards, eg in hurricane zones notorious for centuries, standards such as one does NOT see in New York, would be a good idea, in case a category hurricane like 1938 or 1821 comes in.
But something tells me that’s not what this is about. You only have to consider the grotesque bureaucratese of the testimonies to guess what is really going on.
I’m awardin’ yer a plus one fer
‘grotesque bureaucratese of the testimonies…’
has a nice ring to it, the ring of truth.
Indeed. It’s nauseating to contemplate the internal mental landscapes of those who actually think like that.
Pingback: Weekly Climate and Energy News Roundup | Watts Up With That?