by Judith Curry
There is growing interest in the scientific, operational and applications communities in developing forecasts that fill the gap between medium range weather forecasts (up to two weeks) and seasonal forecasts (3-6 months).
Weather forecasts on timescales of days to season can be used to reduce vulnerability to weather variability as well as capitalize on opportunities. Substantial progress has been made in recent years on the development and applications of medium-range weather forecasts and seasonal climate predictions. However, forecasting on the subseasonal time scale (two weeks to two months) has received much less attention, in part because this time horizon has been considered a ‘predictability desert’. From the perspective of business, weather forecasts on the sub-seasonal time scale provides an opportunity because it lies between the well-established application of daily weather forecasts and the increasing use of seasonal forecasts. Many decisions fall into the intervening two-weekly to two-monthly time scale, so the application of subseasonal forecasts provides the potential to augment actionable forecast information.
Many numerical weather prediction centers now use coupled ocean-atmosphere models to produce ensemble forecasts on the subseasonal time scale. There is now a significant opportunity to develop methods that use subseasonal forecasts to provide actionable information. Probabilistic forecasts can be used to develop decision rules and hedging strategies, identify risk of exceeding critical thresholds, and support cost/loss scenarios and analysis. Actually realizing the potential value of such information, however, depends on the sensitivity to particular weather events, their capacity to act to avoid losses or enhance benefits, and the ability of probabilistic predictive information to influence their decisions.
Under the auspices of the World Meteorological Organization, there is a new initiative Subseasonal to Seasonal Prediction Research. A reader’s digest version can be found [here].
From the Executive Summary of the WMO doc:
The subseasonal to seasonal timescale provides a unique opportunity to capitalise on the expertise of the weather and climate research communities, and to bring them together to improve predictions on a timescale of particular relevance to the Global Framework for Climate Services (GFCS).
For NWP forecasts, model error is not usually so dominant that a reforecast set is needed but for the subseasonal to seasonal range model error is too large to be ignored. Therefore an extensive reforecast set spanning several years is needed to calculate model bias, which in some cases can also be used to evaluate skill. Careful calibration and judicious combination of ensembles of forecasts from different models into a larger ensemble can give higher skill than that from any single model. Comparing, verifying and testing multi-model combinations from these forecasts, quantifying their uncertainty as well as the handling of such a massive dataset will nevertheless be challenging. An important aspect will be to promote use of these forecasts and their uncertainty estimates by the applications community.
Needs and Applications
From the WMO doc:
Weather and climate events continue to exact a toll on society despite the tremendous success and investment in prediction science and operational forecasting over the past century. While many end-users have benefited by applying weather and climate forecasts in their decision-making, there remains ample evidence to suggest that such information is underutilized across a wide range of economic sectors. This may be explained partly by the presence of ‘gaps’ in forecasting capabilities, for example at the subseasonal scale of prediction, and partly by a lack of understanding and appreciation of the complex processes and numerous facets involved in decision making.
In the context of humanitarian aid and disaster preparedness, the Red Cross Climate Centre/IRI have proposed a “Ready-Set-Go” concept for making use of forecasts from weather to seasonal, in which seasonal forecasts are used to begin monitoring of subseasonal and short-range forecasts, update contingency plans, train volunteers, and enable early warning systems (“Ready”); sub-monthly forecasts are used to alert volunteers, warn communities (“Set’); and, weather forecasts are then used to activate volunteers, distribute instructions to communities, and evacuate if needed (“Go”). This paradigm could be useful in other sectors as well, as a means to frame the contribution of subseasonal forecasts to climate service development within GFCS.
Examples of possible applications/users include: warnings of the likelihood of severe high impact weather (droughts, flooding, tropical and extratropical cyclones etc.) to help protect life and property; humanitarian planning and response to disasters; agriculture and disease planning/control (e.g., malaria and meningitis), particularly in developing countries; river-flow and river-discharge for flood prediction, hydroelectric power generation and reservoir management; landslides; coastal inundation; transport; power generation; insurance.
JC note: From the perspective of my company Climate Forecast Applications Network, I am particularly interested in business applications of the sub seasonal forecasts. Here are some sectors that we are investigating:
Energy sector. The nation’s energy companies comprise the primary sector engaging the private sector meteorology industry. Weather is a primary driver for commodity prices in energy, having an impact on both energy production and consumption. Improved forecasts on subseasonal timescales would support hedging for anticipated energy demand, managing and protecting distribution and transmission infrastructure, and weather related energy trading opportunities and risks. The growth of renewable energy is providing new challenges and opportunities for applications of weather and climate forecasts. As of 2013, renewable energy made up 12.9% of the U.S. domestically produced electricity. However, goals of 80% clean energy production for the U.S. by 2035 imply substantial increases in hydropower, wind and solar energy production. On subseasonal timescales, probabilistic predictions of wind, solar and hydropower generation can help stabilize energy costs and supply by improving scheduling and trading, maintenance scheduling, reducing curtailments and imbalance penalties, improving decisions about reserve energy sources, maximizing grid integration, and planning capacity commitments. Specific groups from the commercial sector that would benefit from subseasonal forecasts include energy trading firms, regional power generators/suppliers, and investors.
Agricultural sector. Weather forecasts support operational decision making on the timing of cultivating, irrigating, spraying, harvesting. Seasonal forecasts support strategic decisions regarding crop cultivar selection and intended acreage for planting. Subseasonal weather forecasts present a specific opportunity to bridge the gap in these two time frames. Viable forecast information beyond the traditional 10 day window can extend the time horizon for agricultural commodity price analysis and forecasting, and so support farmers’ decisions about production, storage and marketing, as well as logistical decisions in dealing with regional shortfalls and excess product availability. For commodities with futures markets, subseasonal weather forecasts can support hedging strategies. Futures, forward contracts and hedging are a prevalent practice with agricultural commodities. A subseasonal decision support system has the potential to help users better navigate what is often a volatile agricultural commodity marketplace and reign in risk exposure faced by agricultural producers and suppliers. It also has the potential to help various participants in the agriculture cycle more intelligently join in the appropriate risk management markets via the extension of reliable outlooks beyond the current limited time scales.
Retail sector. I have given this sector less thought, but I would imagine there would be applications for stocking retail stores, particularly stores like Home Depot who need to stock for hurricanes, snow fall, plus timing of seasonal changes. There’s a company Planalytics that focuses on this sector, some text from their web page:
The weather is affecting your business more than you realize and continues to do so each and every day. However, attention to the weather almost always ends up being reactionary and temporary. Removing weather volatility from last year’s sales typically returns 10-40 basis points of total topline revenue (through the reduction of inventory carrying costs and lost sales alone). This means a $10 billion department store chain could capture $25 million in annual savings; a $900 million specialty retailer can easily save $3 million each year; and so on. Last year’s weather is “baked into” last year’s sales data and it’s costing you money. It’s an unintentional error that leads to unwanted surprises. When you look at your business as a whole, the weather only repeats itself year-to-year about 15-20% of the time. If the market-by-market, week-by-week weather volatility is left untouched when building demand forecasts, you are essentially expecting last year to happen again. It rarely does. With Planalytics’ Weather-Driven Demand (WDD) calculations you can “deweatherize” your sales history, knowing exactly when, where and how much weather impacts sales before planning for the selling season.
Sources of predictability
From the WMO doc:
Short to medium-range weather prediction is considered to be mainly an atmospheric initial value problem. The estimated limit for making skilful forecasts of mid-latitude weather systems is about two weeks, largely due to the sensitivity of forecasts to the atmospheric initial conditions. Subseasonal predictions, on the other hand, benefit from both atmospheric initial conditions and factors external to the atmosphere, such as the state of the ocean, land, and cryosphere.
Processes internal to the atmosphere including the Madden-Julian Oscillation (MJO) and low-frequency atmospheric patterns of variability also contribute significantly to the predictability. Furthermore, in a subseasonal forecast, some kind of time average (e.g. weekly or pentad mean) is usually used, which removes part of the weather noise. Therefore, it is reasonable to expect subseasonal forecasts i.e. beyond the traditional weather forecast limit of two weeks, to have useful skill. At this time range the forecasts have to be probabilistic.
Sources of subseasonal predictability come from various processes in the atmosphere, ocean and land, although they are not yet fully understood. A few examples of such processes are:
1) The MJO: as the dominant mode of intraseasonal variability in the tropics that couples with organized convective activity, the MJO has a considerable impact not only in the tropics, but also in the middle and high latitudes, and is considered as a major source of global predictability on the subseasonal time scale ;
2) Soil moisture: memory in soil moisture can last several weeks which can influence the atmosphere through changes in evaporation and surface energy budget and can affect the forecast of air temperature and precipitation in certain areas during certain times of the year on intraseasonal time scales ;
3) Snow cover: The radiative and thermal properties of widespread snow cover anomalies have the potential to modulate local and remote climate over monthly to seasonal time scales
4) Stratosphere-troposphere interaction: signals of changes in the polar vortex and the Northern Annular Mode/Arctic Oscillation (NAM/AO) are often seen to come from the stratosphere, with the anomalous tropospheric flow lasting up to about two months
5) Ocean conditions: anomalies in SST lead to changes in air-sea heat flux and convection which affect atmospheric circulation. The tropical intraseasonal variability (ISV) forecast skill is found to be improved when a coupled model is used.
Teleconnections- Forecasts of opportunity
From the WMO doc:
Extratropical weather is frequently influenced by recurring circulation patterns, usually referred to as flow regimes or modes of variability. Examples of such circulation patterns include the Pacific-North American pattern (PNA), the North Atlantic Oscillation (NAO)/Arctic Oscillation (AO), the East Atlantic (EA), the West Pacific (WP), and the tropical/Northern Hemisphere (TNH). The circulation patterns are usually associated with global teleconnections as in many cases propagation of Rossby wave trains is involved and the atmospheric variability in one place is related to a forcing in another. Because of their large scale and low-frequency nature, the circulation patterns contribute greatly to the atmospheric predictability on the subseasonal time scale.
The strength of planetary-scale teleconnections with both ENSO and the MJO and other sources of subseasonal and seasonal predictability raise the possibility of important windows of opportunity for skilful subseasonal to seasonal forecasts when and where these teleconnections are active and interacting. Such targeted “forecasts of opportunity” would represent a departure from the usual practice in seasonal forecasting where skill levels are averaged across all reforecasts for a particular season and start date, and might spawn a substantial research effort needed to properly represent and convey the conditional skill of such forecasts, perhaps in terms of spread-skill relationships.
JC note: to use a poker analogy, the secret to winning is to know when to ‘hold’ vs when to ‘fold’ (i.e. potentially actionable forecast information versus no useful forecast for that particular period).
JC comments
For some background on ensemble weather forecast, see my previous post How should we interpret an ensemble of models. Part I: Weather models. Also, Wendy Parker has written a very nice non technical article that explains ensemble weather and climate forecasts [link]. If you are looking for a more technical explanation, here is an overview paper by Tim Palmer [link].
I think that focusing on the sub seasonal time scale is important for several reasons:
- there is untapped predictability, with potential for substantial socioeconomic benefits
- to better understand and predict at the timescales of climate change, we need to work up ladder of the timescales, and figure out how to predict at the sub seasonal and seasonal time scales.
The question then becomes how the scientific and funding priorities should be rebalance to to bring a focus to this time scale. I wrote a previous post on this: Climate versus weather prediction: should we rebalance?
My motivation for this post at this time is that I am in the process of writing a proposal about applications of subseasonal weather forecasts. I would appreciate hearing about any ideas you have regarding applications.
Global Warming comes to Minnesota:
http://www.startribune.com/local/287449321.html
Sounds like a potential customer for CFAN.
I would say that’s weather not climate change but I already got the lecture on how global warming causes freezing in the prairies by disrupting wind patterns so more cold air gets drawn down from the arctic warming it more so ice melts more, or some such folderol.
Here’s the problem of using a small scale location thats cold to deny Global Warming. And the reasons its called “Climate Change”.
Remember, the US is less than 2% of the earths total surface.
http://data.giss.nasa.gov/tmp/gistemp/NMAPS/tmp_GHCN_GISS_ERSST_1200km_Anom11_2014_2014_1951_1980/nmaps.gif
You’re also making an assumption that the data quality world wide is as good as it is in the US. My understanding is that outside specific areas, like the US and UK monitoring stations and records are much more spotty and satellite records are so recent you can hardly claim to identify a climate trend from them.
Extrapolating from the US alone may be less then ideal but pretending we have high quality long term data across the entire globe seems equally foolish to me.
What about insurance companies? Customers with short-term info regarding upcoming weather could well have interests in purchasing, and there’s no way an insurance company could give a competitive rate without the best forecast they can get.
Insurance/reinsurance companies look for forecasts a year in advance. I have tried to generate interest in using sub seasonal/seasonal but no success
Could your company get an insurance protecting it against financial consequences of incorrect forecasts?
Emergency management types like fire and police could use some of this too. Same thing with municipal services types. It would a lot easier to plan vacation time if you know when the cold snaps are coming and you need lots of people on to handle frozen water pipes.
It might be useful for balancing flood control and water storage, e.g. when and when not to drain reservoirs.
It is my understanding that different forecasters consistently make seasonal forecasts that are inconsistent with one another. Has there ever been a seasonal forecast where all the forecasters agreed (and it came true)? This would seem to show that there is no predictive skill and I expect the same from subseasonal forecasts. The fact that every forecaster is sometimes right does not demonstrate skill.
Making the forecasts probabilistic makes things even worse because a probabilistic forecast usually cannot be falsified, so there is no way to measure skill. Nor is decision making probabilistic.
Discussion of the utility of subseasonal forecasts assumes that accurate forecasts, which (almost) every forecaster agrees to, are possible, which seems unlikely. Keep in mind that believing a false forecast is often worse than believing none, and can even be catastrophic. Saying that a given forecast is not possible might be very useful, but ironically there is probably no money in doing that, which is why it is seldom done.
what they said
“However, forecasting on the subseasonal time scale (two weeks to two months) has received much less attention, in part because this time horizon has been considered a ‘predictability desert’. ”
what you said
” This would seem to show that there is no predictive skill and I expect the same from subseasonal forecasts.”
“Making the forecasts probabilistic makes things even worse because a probabilistic forecast usually cannot be falsified, so there is no way to measure skill. Nor is decision making probabilistic.”
All prediction is probabilistic. All measurement is probabilistic.
All decision making is probabilistic.
you probably thought you were saying something smart.
probabilitistic forecast models can be confirmed or disconfirmed.
we dont confirm or disconfirm predictions or forecasts we confirm or disconfirm the system that made them.
I have no idea what you mean by probabilistic. I know of no one who calculates probabilities when they make measurements or make decisions. Buying groceries would take forever. This is a mistaken metaphor at best.
Mosher wrote:
“we dont confirm or disconfirm predictions or forecasts we confirm or disconfirm the system that made them.”
Please describe how you validate a model without first evaluating whether the output reasonably tracks the system being modeled.
Mosher wrote:
“All measurement is probabilistic.”
How many cars did you simultaneously drive from home to work today? I’m envisioning the PDF on this, just wondering if you could confirm.
“I have no idea what you mean by probabilistic. I know of no one who calculates probabilities when they make measurements or make decisions. Buying groceries would take forever. This is a mistaken metaphor at best.”
When I buy groceries I take every cost and round it up. I’m pretty sure I do this correctly, but sometimes I may make a mistake. So, Im not certain, but more often than not I round up correctly.
So, moving through the isles I put a 1.58 thing in the basket and say
“2” When I get to 50 dollars I decide to stop. I make this decision because
I have 60 bucks in my wallet and I figure that I’ve probably done the addition correctly, and I’ve probably remembered the number correctly,
and that even if I havent I probably have enough in my bank account to pay by debit card if I have to. I also hope the cashier rings me up correctly. Most times they do. So while I am not certain I have enough money, I probably do. I dont have to calculate the probability to know that
A) my decision is based on probabilities
B) I could be wrong.
Buying groceries takes no more time doing it this way and it certainly doesnt take forever, so your prediction is wrong. Your mistake was thinking that one had to actually do a calculation.
When I make a measurement it always comes with uncertainty or error.
This means the true answer is probably within with bounds I specify.
Keep in mind that decision making and decision theory is my field. Lately I have been writing about decision making by horses and other animals, which turns out to be pretty complex. See my http://horsecognition.blogspot.com/. Do you think horses calculate probabilities?
Probabilistic models of decision making are common, but it is wrong to think that means decision making is itself probabilistic. To do so is to confuse the model with reality, something we also see a lot of in climate science.
David you have not done any interesting work in decision theory.
Even worm brains calculate probabilities.
Now, the CONSCIOUS calculation is another thing.
in order words just because you are not aware that your are calculating a probability, doesnt mean your are not.
have a look at openworm
Steven Mosher – you seem to avoid scenes of your past crimes. It is off-topic here, but I don’t know how else to reach you. I actually believe that you know a lot about the subject, but please don’t refer me to a Powerpoint presentation.
ERL – Effective Radiative Level. How high is it in Berkeley now? Does it change with the time of day? With seasons? With a location? What is the temperature there? Can your equations predict the ERL temperature?
Mosher
Get a debit card
nottawa, I was going to suggest a debit card too, but I do something similar to what Mosher described. For example, when I have a coupon for 15% off if my purchases exceed $100, I round the price of each item down (e.g., $15.49 becomes $15.00) in estimating the sum until it has passed $100.
Max
I criticized my wife for years for not using cash. It took her longer checking out when I was with her and others in front of me seemed to slow things down. But when my bank offered money for using my card, I switched. It is strange what a person will do for $5.40 per month.
Steven,
“Even worm brains calculate probabilities.”
No! Brains LEARN-from-experience.
If the aforesaid brains happen upon a lot of extremely low probability events, then that’s what they think will happen. It keeps the worms happier, not knowing there’s anywhere better to live.
nottawa rafter
“I criticized my wife for years for not using cash. It took her longer checking out when I was with her and others in front of me seemed to slow things down. But when my bank offered money for using my card, I switched. It is strange what a person will do for $5.40 per month.”
“…strange what a person will do for $5.40…”
In behavioral economics it is known as the power of free. We are predictably irrational and we will respond to the perception of a free lunch. My guess is that it is a successful adaptation that drives us to maximize EROEI. Google, Facebook, etc exploit this tendency, except their services are not really free – the currency is personal information, which they sell, if only in an indirect way.
Aang there is a difference between learning and making decisions, although decision making does involve what you have learned in the past.
I have this probabilistic thinking in common with Mosher. I try to convince other people its a great way to make decisions, especially those people I see who keep making bad decisions over and over….
@Wojick I think the way to verify probabilistic forecasts would rely on having enough of them run to verify the distribution of the outputs. Which, of course, would then be possible for these short runs…
Perhaps there is a more subtle point you are getting at? I missed it….
“All measurement is probabilistic.”
LOL.
You must be the most disingenuous poster on any blog anywhere!
interestingly, all the seasonal forecasts last March/Apr were for a strong El Nino. This was surprising, because usually the models (and forecasters) don’t show much agreement
There are serious ways to evaluate probabilistic forecasts:
In the meteorological literature there are several methods for assessing the value of probabilistic forecasts, including the Brier Score, Ranked Probability Score, Relative Operating Characteristics (ROC), Bounding Box, Rank Histograms, and information content.
The objective of a probabilistic forecast is that the forecasts should be correct in a probabilistic sense, e.g. in the long run the 90% quantile should be exceeded by the actual value in 10% of the cases.
Hi Judith,
Weatherbell went on record early last spring that there was very little chance of a strong el Nino.
I know there are ways to evaluate long runs of forecasts (assuming one can actually define the long run) but I do not think any actually measures skill because none of the forecasts are wrong. Probabilistic forecasts are not forecasts because they cannot be wrong (except at the two extremes). They just sound like forecasts and this is a major conceptual confusion.
I would guess that’s only part of the objective, the other part being to get the 90% range as narrow as possible without violating the above.
David, since this is your field I wonder if you know of a way to assess probabilistic forecasts. I have been thinking about this ever since engaging Willis E at WUWT about his assessment of Piers Corbyn, someone who makes his living by selling his forecasts that include the sub-seasonal range. While I remain skepitcal as well, I didn’t like Willis’s reasoning.
The problem is the probability so far out has to include time and space. How do we rate a forecast? It was the right description of the (unusual) weather, but it turned up 3 days late or early. The weather turned up at the right time but in the wrong place. The weather turned up at the right time and place but it was rain rather than snow etc etc.
Willis made the correct point that if you make enough guesses eventually some will be right, but that doesn’t really tell us whether a forecaster is doing better than chance. He maintained that if you made a forecast, then to be deemed “accurate” it had to be exactly as forecast. If you are out by a day or two or by 100 miles then you are wrong.
I was thinking that the analogy could be like a dart board. A bullseye is getting your forecast right in all particulars, but the further you are away from the board the harder it is to hit. The concentric rings around the bullseye reflect the diminishing accuracy but still a demonstration of skill. Surely it is not beyond the wit of man to devise some sort rating system whereby points are awarded for distance from board, and closeness to bullseye, reflecting advance prediction and spatio-temporal accuracy?
My second point relates to your justifiable skepticism that because forecasters can’t agree on a prediction, and eventually they will be right some of the time, then there is no reason to take notice of any of them. But what if one of them was right a lot more than chance? If you are not looking closely at each individual performance then you will miss an opportunity to find a methodology that is skilful or useful. Also there is the matter how to rate whether their predictions are accurate or not. Presumably we can safely say if they predict lots of snow and clouds and it turns out to be mild and sunny they have missed the board entirely, but what about a little snow and clouds…..against which you have to deduct the likelihood of there being snow and clouds for that time of year or location. You get the idea.
Forecasts of long range will always be a best guess situation. The planet when understood, is in constant change from water loss to space to distance changes from the sun…but a general assessment of areas for decades is possible with watching salt ocean patterns as that dictates evaporation. Huge velocities differences have never been studied and water changes directions at the 48° latitude due to velocity difference and centrifuge forces.
Dr. Curry,
Aren’t researchers for the Commodities Markets already deeply involved in this area? I assume the research is generally proprietary, available to the public, if at all, for a fee. Orange juice futures jumps to mind as a potential gold mine for the researcher who can produce an accurate forecast about hard freezes.
Yes commodities markets are big users of weather/climate info; however there hasn’t been much from the commercial sector on sub seasonal timescale
I will be looking at adding a weather forecast to a stocking model.
John
Living in England I follow the daily met office forecasts closely and also their five day forecasts. Their big problem is in their not getting the detail right for micro climates and their timing of weather events
I live 15 miles from their Exeter office on the coast and they often fail to get our weather right. Their new 100 million pound computer is designed to overcome this problem and in particular to forecast heavy rain events in the narrow valleys that are prone to flooding the towns at the bottom, such as Boscastle.
The bigger problem though is the timing. Rain during the day followed by a clear night becomes a different thing entirely if the day turns out to be clear and the rain comes at night. This greatly affects the local tourism industry.
That could be a problem with the orange forecasts you mention. Get the weather window eight hours wrong and the overnight rain followed by a sunny day becomes overnight clear skies with a possibility of frost and rain during the day
Tonyb
John Vonderlin – “…Orange juice futures…”
Too late, Clarence Beeks already nailed this one.
Skip the advert.
http://m.youtube.com/watch?v=RLySXTIBS3c
John V.
I don’t know about the market for longer range weather forecasts, but I imagine it’s rather like the investment adviser/mutual fund industry. Fund managers and investment advisers are ranked by those in the industry who make money by ranking investment advisers and fund managers. The top ranked guys get the client’s and the money, until the new rankings come out, then the new top ranked guys get the client’s and the money.
Which is a good reason to invest in index funds rather than actively managed funds.
The rankings are usually based on how lucky the advisor gets. Nobody can do better than the market in the long term because there is no way of knowing whether all information known about a firm has or has not already been incorporated in its share price.
Daniel Kahneman’s book “Thinking, Fast and Slow” is highly recommended reading but it is to be read slowly and thoroughly as it is quite technical in its approach to the study of the effect of bias in human thinking.
Peter Davies
IMHO, the work of Kahneman and Tversky is some of the most important in the field of human behavior. A more entertaining and easier read is Dan Ariely’s “Predictably Irrational”.
Thanks for the suggested reading Justin. I see that the book forms part of a series by Dan Ariely. I have bought a kindle version for $8 and will get around to reading it shortly.
I am interested in the psychology of human decision making, particularly as it relates to important issues of our times and the way in which they are debated.
Peter, I think you are referring to the Efficient Markets Hypothesis and misinterpreting its meaning. The theory is that you can’t beat the market, because the market is informationally efficient. The price of assets already reflect the information relevant to their value.You don’t know anything the market doesn’t know. You can’t find miss-priced stocks. Don’t even try. In the theory of the strong-form of the EMH, even inside information is reflected in stock prices and stock prices instantly change to reflect new information. Read Eugene Fama, Burton Malkiel and Paul Samuelson, for more detail. I think Samuelson has the more nuanced view.
Hello Don, no, the EMH only relates IMO to the pricing of stocks whereas I was speculating about the decision to trade stocks and about the timing of such trades. In this context the market index, as Max_OK has suggested, is “the market”.
Kahneman suggests that “expert” traders generally do not do better than “the market” and that stock trading algorithms work better than decisionmaking by investors influenced by their emotions of fear and greed in random succession.
Peter Davies
I know nothing about stock picking other than I am very bad at it. I hate to admit it, but I think Max was right about index funds. Don’t tell him I wrote that.
If I had nerves of steel, I would follow Nassin Taleb’s method and buy puts betting that a black swan will come along eventually. It does. The down side is you have to accept losing money everyday until the black swan swoops in.
Justin, I agree that straddles are a common technique for sophisticated investors. By sophisticated I mean that the investor fully understands the underlying risks and accepts that you must necessarily take on higher risks for higher returns.
Everybody dreams about making a fortune on the stock market but are only prepared to do this with minimal risk. This is impossible in a market in which the price of shares have incorporated all the information about each company operating in it.
Please Peter, stocks are bought and sold because of their prices. That’s the motivation. Identify underpriced stocks and buy them, or overpriced stocks and short them. The old buy low, sell high. My recollection is that Kanehman is not talking about the EMH, or whatever it is that you are talking about. He says that most investors are not equipped to beat the market for various psychological reasons, to put it simply. Many professional investors will tell you the same thing. Where is our Nobel Prize?
I agree completely with your last comment Don. EMH was never in my mind because IMO the stock market share prices do not, in practice, perfectly reflect all available information about companies and their trading environments.
The decision to trade is certainly based largely on pricing but other factors obviously influence its timing as well, such as taxation or the need for rebalancing a portfolio.
My original comment that expert sharetraders generally do not beat “the market” seems to have raised questions as to its validity in your mind, but be assured that the EMH was far from my mind when I suggested this.
We are on the same page, Peter. Re. the EMH, most investors would do well to accept that they aren’t going to find any mispriced stocks. However, the strong-form of the EMH is demonstrably false. Those in the know, make the dough. Corporate finance can be very murky, and deliberately so. And then there are the many examples of outright fr@ud. A monstrous example of market inefficiency is the Chinese reverse merger racket. If you are not familiar with that mess, google SINO Forest. There are hundreds of them. People will buy any story that Chinese companies are selling and there is no transparency. Probably $50billion has been lost on these sc@ms in the North American markets and more in Hong Kong and Singapore. Some very sophisticated investors were taken in. I am happy to say that I was among the shorts who exposed them. Someday, I’ll write a book.
Interesting about the Chinese companies, because it is an excellent example of a privileged few benefiting from an undemocratic regime. The mass of the chinese people are not benefiting at all from the boom.
The lack of effective corporate regulation in China is well known in the Western World, especially in respect of ongoing patent and copyright infringements, and the scams you mention Don comes as no surprise.
When you get around to writing your book, it would be of general interest to many investors to find out how you were able to detect that all was not well and bail out as you did.
It’s a long story Peter, but while you wait for the book I’ll give you a quick summary. I never invested in companies based in China or did any business there. Like everybody else in the 1990s, I was interested by the China “story” and went to have a look around. I saw endemic corruption and a government that was not going to protect the interests of foreigners. Look what they do to their own people.
So I ignored all the hype about the China miracle and the piles of money to be made there, until I became aware of a few smart guys (google Carson Block), who were investigating Chinese companies that had gotten listed on U.S. and Canadian exchanges by doing reverse mergers with shell companies (RTOs). The problem with this is that they were able to get listed by filing documents/financials with our regulatory authorities that were not verifiable. The Chinese govt. doesn’t allow our SEC etc. to make any investigations in China. But those of us who know how to find things out by spending a little money were able to get the filings made by Chinese companies to the bureaucracies that collect taxes and hand out business licenses. What we found was that nearly all of the Chinese reverse merger companies were making ridiculously false filings to the exchanges and to regulatory agencies. To make sure that the companies weren’t making false filings to the Chinese authorities instead of to ours, some of us hired people to watch their operations in China. Their “plants” were observed to be Potemkin villages. Again, see the SINO Forest story.
So we short the Chinese RTOs and spread the word that they are worthless. Up to now there are a lot of losers who still won’t believe they were taken. Some very high profile billionaire investors, including Hank Greenberg and John Paulson, were taken in. Google “Wall Street Scion Lost in China Agritech”, the link has a word that will trigger moderation. I confronted these scions on various stock discussion message boards and they tried to intimidate me with threats of legal action and calling the FBI to investigate my allegedly criminal behavior. Losers with more money than brains. The old man subsequently sold his investment company, rather than turn it over to his scions.
And this mess still has not been cleaned up. The SEC is useless. The message is, you are on your own with investing.
I have a comment in moderation, Peter. I tried to avoid all the “bad” words, but for some reason it still went to mod. It’s long, so if it doesn’t appear, you will have to wait for the book.
Thanks for your response Don. I only just went back over the old threads to make sure that I have at least covered all posts that have been directed to me and picked up yours. Apologies for that.
The way you spoke in your previous comment it sounded as if you had actually invested in Chinese companies but had short sold them when you realised that all was not well.
Investing is hazardous enough without having to contend with corrupt governments and the general lack of transparency in the economic environment that China entails.
To All,
Sub-seasonal hedging in the commodities markets for derivatives linked to energy or agriculture (including livestock).
Sub-season route choices for long-distance shipping of anything, especially by ocean-going vessel (it is about a 2-week trip from Shanghai to Los Angeles, for example).
In the U.S. retail arena, sub-season route choices and shipping dates for the year-end shipping crunch (both for stocking stores in advance of Black Friday post-Thanksgiving and for X-Mas deliveries to consumers). This overlaps with route choices for long-distance shipping if the goods are manufactured overseas.
One-two month early warning for storm surge/flood protection risks, especially during hurricane/typhoon season.
Better resource availability for infrastructure repair required due to extreme weather (e.g., human and physical resources for water main breaks and road repair due to extended freezes, and better stockpiled availability of salt and de-icing chemicals for previously unpredictable extended snow and ice events).
Regards,
MK
Well you may already be aware of this guy. He does have one paper or lecture toward the very bottom in 2010 about subseasonal and pests in Switzerland but the whole of the papers seems along the lines of what your talking about:
http://www.ufa.cas.cz/dub/dub.htm
Outdoor wedding planners. But they would probably want to see your model’s track record, before shelling out. Newsletters predicting various things, often are profitable. An ensemble of newsletters usually works best.
Judith, I an trying to think of uses for 1-2 month forecasts on my farm. Not coming up with much.
Short term weather forecasts are essential for planting, spraying, haying, harvest decisions. Where folks have center pivot irrigation, whether to turn the system on or wait for rain.
Seasonal forcasts help decide things like cultivar selection ( maturity length especially).
One to two months? Not many decisions depend on that time frame.
Selling/buying futures as a hedge?
It is in the range of a typical contract held.
AK, no reasonable farmer hedges on less than a growing season. Banks won’t let us play negative zero sum commodity games that way. Such hedging is for the trader gamblers on CMX who would otherwise be playing in Wisconsin’s Indian casinos.
Explanation. The net result of all futures contracts must be zero, less transactions costs. Farmers know this, and seldom hedge. Rather, we decide how much to plant (manipulating supply) and when to sell (manipulating demand). The latter accomplished via ‘grain bins’ either on farm or at local country elevators.
Translation. Farmers are like ‘native americans’. We own the house, so can set the odds.
That said, forecasts that help us farmers improve yields are welcome. So long as everybody does not benefit and there is no glut.
No commercial products for that time frame .. but never underestimate Joe Bastardi. What makes modeling that time frame special? Is there a need for more accuracy than for seasonal forecasts?
Ahem. Sorry folks. Governments. From energy demands, water supplies, planning of any number and types of projects (when to schedule shutdowns for maintenance), road and bridge repairs, staffing.
I was going to say tourism (like tonyb) but that’s a legal minefield.
Test cricket. A test match is played over five days and there are some very heavy punters.
The increase in seniors provides more opportunities for impulsive travelling based on the probability of good weather. Resorts in the south could develop a multi faceted marketing strategy targeted to seniors indicating great outdoor recreation weather expected in the next 1-2 months. Vacation destinations send out ads continuously but the uncertainty of nice weather certainly hold back many who don’t want to take a chance too far out. I believe impromptu vacation planning will be rising in the next few decades, held back only by unknowns surrounding weather.
“Get ready for some bad weather,” the man told his neighbor. “How do you always seem to know before it happens,” the neighbor asked. “Easy,” said the man, “the fellow on the corner goes off for a couple of months every year at about this time and he usually takes is umbrella down before leaving; but, this time he left it up, which means big storms are coming and that umbrella will be a goner.”
Many of us on the land would love it but are deeply skeptical. Climate is complex, computers only think they’re complex.
There is also the concern that an expensively acquired forecast could over-ride commonsense, observations and radar in emergencies.
You have to avoid the growth of an internal Anti-Whoops industry: a creative branch of the science devoted to explaining how forecasts were right but wrong, right in principle, wrong but for good reasons, right in a counter-intuitive way, right within parameters of uncertainty, right next time…and so on. Don’t tell me 70% chance when you think something is going to happen. Customers will get mighty tired of that dodge behind a comfy 30% buffer. If you set up shop, the merchandise has to be sound. Australia’s BOM already gives out seasonal rubbish for free.
Wrong means Wrong would be a good motto for an ambitious and expensive forecasting operation. But after one spends a hundred million pounds on computers and with insurance and agricultural companies breathing down one’s slender neck…
To paraphrase the Godfather, if we’ve learned anything from recent history, it’s that you can spin anything.
mosomoso wrote:
Don’t tell me 70% chance when you think something is going to happen.
I watch the forecasts and I watch what happens. I think their 70% forecasts are really good. They show me the radar and I can second guess. When we don’t get the conditions that were forecast, someone around us usually does. Sometimes it rains around us and not here and sometimes it rains on us and not around us, but there is usually rain somewhere in Houston when they forecast 70%.
It may not work that well where you are, but it works in Southeast Houston, Texas.
If someone gives away a short-term forecast, with help from observations, radar, sat and breeze sniffing, no probs with 70%, though I’d prefer conversational English. Our BOM tends to miss in the margins as often as it hits, but at least it can see a huge blob of weather coming from the West.
But a paid seasonal forecast for conditions which have not emerged? With sowing, cropping, water releases, tourism etc at stake? Keeping in mind that our history is full, in any case, of contradictions re effects of ENSO etc. Lethal 1939 deserves to be the El Nino of El Ninos in Eastern Australia – except it was a La Nina flanked by neutral years.
Not saying it’s impossible or undesirable. But it’s a hugely ambitious punt. So much comes back to the notion that because there SHOULD be a mature science of climate therefore there IS a mature science of climate. Does not follow.
sub seasonal forecasts. This is really good.
These are forecasts that we can compare with real data, in our lifetime, to see if they will show skill. We can see who is right and who is wrong. We cannot rule out luck until we see consistent results and reasonable explanations.
Climate forecasts are for the future and very time they are wrong, they just move the target. That needs to be stopped.
There are long term cycles that they do not understand. It looks like they don’t even suspect.
Stratosphere-troposphere interaction: signals of changes in the polar vortex and the Northern Annular Mode/Arctic Oscillation (NAM/AO) are often seen to come from the stratosphere, with the anomalous tropospheric flow lasting up to about two month
Upper atmospheric aeronomy has often external drivers,that have effects on the persistence of climate structures such as NAM ,and SAM.
http://www.nature.com/ncomms/2014/141014/ncomms6197/full/ncomms6197.html
References 17-20 also.
It seems to me that a sub-seasonal forecast would best be used as an update to the seasonal forecast.
Using the Home Depot example, they’d love to know 6 months in advance how many snow shovels to have in stock, but they’d also like to know 6 weeks in advance if their July decision on snow shovels was the right one or they need to place an order.
If they have good reason to believe they get decent seasonal forecasts with a sub-season updates within a reasonable planning period, it can still save them a bunch of money. A seasonal forecast that says “snowy” means you buy 10% more snow shovels at July prices. A sub-seasonal forecast that says “January and wow is it going to be big!” means you can order a few more at December prices along with some sleds and snowblowers and plan your post-Christmas displays around snow equipment.
My earliest familiarity with sub-seasonal and long range weather forecasts was the hoary Farmer’s Almanac. Being decades ago, before computer model usage was common, I suspect they just used back-of-the-envelope log-a-rhythms. Overall, their accuracy was significantly better than a broken clock. However many a spring day, the planting warmth they had predicted had to be produced by enjoying a glass of wine by the woodstove as we watched the snow fall. The missus and I found that strategy of dealing with their misses more palatable than just taking their predictions with a grain of salt.
Judit, i would like to see a comparison with what was forecast and what was observed. Do you have example that show actual skill?
Dr. Curry, can you see the day when you can predict sea surface temperatures near the coast 60 days ahead of time? I live in a touristy area and businesses here could tout water temperature in 60 days as a means to get tourists to make plans to come visit. The desired prediction window would be mid April to mid June, with the predictions made 60 days ahead of time. The geographic sector would be the Spanish coast from say Torremolinos to Castellon, and of course Ibiza and Mallorca. I live on San Juan beach, and the business community here would kill if they could show when the water reaches 22 degrees C.
thx, this is one i hadn’t thought of
Are you serious, Judith? Everybody already knows the waters are going to get warmer mid April to mid June. The water temp will reach 22C, about the same time it does every year. During the tourist season. And if it doesn’t reach 22, it will be in the ballpark. If the water’s colder than normal in late May, it might take longer than usual. Ask the friendly neighborhood fishermen, beach lovers, and the local weatherman. They will tell you for free.
If some Stanford prof. came to me with an idea like this in my VC days, I would have told him/ her to come back after he/she had solved the nasty predictability desert problem.
My guess is the appearance of the accuracy of the 2 week to 2 month forecasts can be improved by making them sufficiently vague. Just a little less vague than the seasonal forecasts is probably the way to go.
But the key is to have a better idea of the future weather. The way i see it we could have one of those LED jumbo screens somewhere in London, with the TV camera pointed at the beach, and current conditions (we already have one on the Internet), then we add two sets of information, “water temperature in 30 days” and “water temperature in 60 days”. The key is to have credibility, and give all those Britons time to make reservations to fly here. I don’t think the city here knows how to market the beach for the shoulder seasons, and this type of approach could help.
But Fernando, what does it do to your beach resort’s credibility, when the people come and the temperature ain’t as advertised? Do you fall back on the predictability desert excuse?
Well, ain’t nobody else can tell what the water temperature gonna be in a couple munts? Don’t believe everything you read on those jumbo screens.
Do you really want a lot of pale fat people on your beach, or are you funnin us?
Don, the advertisements can have small print with the usual disclaimers. Or they can be expressed as probabilities. Do they want tourists here? Sure they do.
In my case I prefer more tourists in the “shoulder season”, say April and May. This gives me a slightly better chance to find top chefs and higher quality food in local restaurants. The area infrastructure is set for July and August, so if we get 50 % of the peak load the beach can handle it (we have over 15 km of really good beaches). The extra influx helps the local economy, and some of my neighbours would benefit.
I went to high school in Florida and I notice their cities do a much better marketing job. But that’s an American strength.
I sympathize with your restaurant problem, Fernando. I just don’t think you are going to get any help from the predictability desert. Spanish food is my favorite and the restaurants around here are not so good. I do it myself. Learned how to create good stuff by watching Spanish chefs on youtube. Jose Andres has a lot of good videos. Most people who have eaten my tortilla de patatas say it’s better than any they have had in any restaurant. I served tortilla, zarzuela de mariscos, albondigas with romesco sauce, various other tapas and crema catalana to guests on New Years day. Big hit. Cuban food is good also. Why don’t you do a Cuban restaurant, for the touristas? You are Cuban?
Thank you Judith for this post. Money spent on global climate prognostications would IMO be much better spent on the improvement of shorter term weather forecasts for vulnerable regions.
Recent natural gas price activity seems to indicate a need for sub seasonal forecasting. Prices dropped significantly through December as wide spread cold spells failed to develope in the contiguous USA. Now with the recent cold prices are moving up again. Traders seem to be doing little more than looking out the window.
Yep
In Australia seasonal and regional forecasters tend to go together. There used to be individuals who were regularly consulted on such matters and were regarded by the media as reliable. Were they academically qualified? { don’t know, or maybe just lucky. There always will be such people of varying credibility and are a part of our noise environment.
can’t help but see a down side for regular folk
good weather time slots go up in price
less able to pay are reduced to the bad weather weekends
:)
seriously, everything nowadays has curious end result being of most benefit to the top**
seems to be a trend
despite good intentions
super computer output puts out mostly for the super wealthy
will be giving up a lot when we can predict the future
**very soon we are going to wake up one fine day and all wealth gains in the US will have gone to the top few %
Always appreciate your ad hom free contributions John but IMO your last sentence has already happened and has been for decades.
not quite 100% yet though. is it?
in Camelot
it sounds a bit bizarre
in Camelot
the rain may never fall ‘ti after sundown
by eight the morning fog must disappear
Not quite 100% of all wealth gains John but IMO the serfs are only getting gains from inflation, with little real value accrued. The real wealth gains IMO have been, are, and will always flow to the top few %.
I think you are right
seems like a Rubicon or a Rhine should be crossed at some point
It’s the same in the Russia and China, too.
wait…
as right wing, Koch brother loving climate change deniers, aren’t we supposed to be okay with it?
Do not read this:
http://www.economist.com/news/briefing/21637355-freelance-workers-available-moments-notice-will-reshape-nature-companies-and
The destruction of distance, the winner-take-all economy, the commoditization of labor, and the race to the bottom are all in play.
The old farmers almanac provides a useful clue. Regional climatological records over a long time are still the best clues, what happened before will happen again, but figure out the timing of the cycles, try it, you’ll like it.
I spent a part of my professional life integrating weather forecasts into offshore oil and gas operations (drilling, construction, operations). I started out providing the forecasts to those operating and eventually moved on to manage offshore operations for a major oil company.
In my experience the most useful actionable forecasts were short term (less than a day). These could facilitate making critical decisions concerning starting or stopping risky operations (running a riser, initiating a platform launch etc.). The uncertainty of longer term forecasts (days, weeks months) combined with the uncertain duration of operations made it almost impossible to rely on forecasts for planning purposes. It seems unlikely to me that the reliability of sub seasonal forecasts would ever improve to the point where a manager would be willing to make $10-100 million decisions based on them.
I know a bit about weather impacts on commodity (power) trading and again the focus is largely on very short term forecasts (which can and should be fairly accurate and actionable) as opposed to weekly or monthly forecasts with high inherent uncertainty.
Putting my global skeptics hat on it seems that maybe the GCM folks are looking for a new source of funding to keep playing their computer games.
Thanks, Mark. That’s an appropriate ending to this thread.
No problem Don. The huxterism of the modelers is one of my pet peeves.
If it hasn’t been done, it probably can’t be done.
JC is wasting her time on this subseasonal forecasting notion. Her time would be better spent doing something that’s already been done, or just taking it easy.
Max, I agree that sub seasonal forecasts are probably a waste of time and money. I guess Judith is looking for business opportunities for her consulting company.
More power to her, but I don’t think this is the way to go.
Computer modelers will lead you down the garden path every time with their pretty graphics and elaborate promises. It took me a couple of years to learn this when I first started out.
Mark, I assume the idea is to improve the forecasts to deliver a useable product. Say you have to plan a tricky platform approach for a jack up, but have rig schedule alternatives. You could make plans to move to a specific platform if you had a better idea, do the required preparations to mate the two structures with a better idea of what could happen. Also think about the Shell tow of that Arctic rig in Alaska, a better long term forecast may have let them know it was better to stay in port and pay the real estate taxes.
Fernando, I guess it’s really a matter of whether or not you believe that the accuracy/reliability of sub seasonal forecasts could be improved to the point where one could really use them cost effectively. My feeling is that you could spend a lot of money trying to improve these forecasts and see only marginal improvements. Been there, done that!
As to your point re. Shell and the Kulluk rig tow I think that there was plenty of useful data (historical and forecast) available to them on what to expect if they made the tow and I am certain that their Warranty Surveyor worked the weather forecast problem to death. My take away from the NYT article this Sunday on Kulluk is that the issues were more a matter of Shell’s internal risk management systems breaking down than the need for better weather forecasts.
I guess we just have a difference in our expectations. Given the amount of cash we spend I wouldn’t mind having a better 60 day outlook. I cooperated with Canmar on long range SSDC tows from Alaska, and I voted to can the effort, I saw it as a pretty risky project. When I visited Alaska a couple of years ago I heard about Shell’s plans, and I thought they just didn’t fear the weather issue enough. Also, the had the wrong people on that project. What a mess.
Logistical, last year there were salt shortages. The companies said they had the capability to produce the salt, but the weather (freezing rivers, roads, demand surge…) made it cost prohibitive to supply the salt needed when needed.
“On subseasonal timescales, probabilistic predictions of wind, solar and hydropower generation can help stabilize energy costs and supply by improving scheduling and trading, maintenance scheduling, reducing curtailments and imbalance penalties, improving decisions about reserve energy sources, maximizing grid integration, and planning capacity commitments.”
____
Renewable sources of power are more important in Europe than in the U.S. Germany’s commitment to renewables is well know. Spain and Denmark rely heavily on renewables. Renewables are now Scotand’s largest source of power.
http://www.scottishrenewables.com/news/historic-milestone-new-figures-show-renewables-now/
What is the European experience with subseasonal forecasting?
Max
Sorry, but the story you linked to regarding scottish renewables is highly misleading and was even taken apart by the former chief scientist of the govt dept of energy and climate change
http://bishophill.squarespace.com/blog/2015/1/5/wwf-spivs-are-spinning.html
Tonyb
Tony, I think you are referring to professional pedant David MacKay. Perhaps I missed something but all I saw at bishophill were his complaints about how “homes” are defined.
I get the impression bishophill is an anti-renewables blog, and I’m not impressed with its deizens. You can get an idea of the kind of readers a blog attracts by clicking on the advertisements. I think many who frequent bishophill are unsuccessful, because I saw an ad that said “How to avoid running out of money when you need it most.”
On a positive note, you and others who are tired of winter, might consider a Caribbean cruise with a unique theme:
https://www.cruisenude.com/
So, as an example of a sub-seasonal forecast, at the beginning of January, we would have to say something about what February and March will be like for a region. Obviously this would not be a day-by-day forecast, more like saying how these months would relate to their climatological averages, e.g. warmer than average, wetter than average, etc, along with some probability of that. This is a very difficult proposition. From this graph we can see that monthly anomalies are hardly even correlated with the previous month for global data, and for regional data it may be even less correlated.
http://www.woodfortrees.org/plot/gistemp/from:2013
Even if there are long-term large-scale ongoing anomalies in the ocean or soil, I think the weather variation would be larger than such a signal, and that is essentially a large random perturbation. The problems here are no better than for seasonal forecasts of 3-6 months, possibly even worse if you want to refine it to months instead of seasons. There is just too much noise in my opinion and the weather determines the monthly average more than any ongoing anomaly.
Regarding renewable energy, we might also want to know if month will be sunnier or cloudier or windier than average. Equally hard because of the weather dependence.
I hate it when jimmy dee and maxie are making sense.
An improvement on probabilistic predictions helps, if the service is charged appropriately, and the statistics it delivers are testable. In the oil industry we have very high costs. I was also wondering if some parameters may not be easier to predict with a tighter range.
@Fernando
Thats where good aposteriori error estimates (as probabilities) would be very useful.
I remember talking to the guys at NCEP about this, and the take away is that the usually had a pretty good idea based on the particular weather patterns and the divergence of the models on how ‘predictable’ a given regime was.
So, in an ideal R&D world, this capability would be developed further.
The model will tell you when it is unstable and when predictability is less/more. One just has to ask!
Nickels, I had my company pay for research on some really goofy things a few years ago, I didn’t mind tossing money at strange ideas. One of these days something will pan out. And we sure can’t get ahead if we think we know and we don’t know. Me, I don’t have the foggiest idea. Whether scientists want to spend time on such an effort is up to them. But I don’t mind pushing the imagination envelope.
While CAGW predictions fail to materialize and mis-guided nations and leaders continue to focus on beneficial warming as if it’s the end of the world and ignore reality, the push for renewables like solar/wind, will end up making our power grid unstable- leaving millions without reliable power during the times when they’ll need it the most-ie.,during very cold winters.
And the military, for many purposes.
Campaign Barbarossa 1941.
Judith,
One application for seasonal forecasts would be to make sure that surfing contests are held during periods of good surf. Large surf is caused by major storms, and if those storms could be predicted, it would ensure that there would be large surf for professional surfers. There have been occasions where major surfing contests have been held in virtually no surf.
JD
JDOhio
We have some big surfing beaches close to us in Cornwall. In practice how would the knowledge of the POSSIBILITY of no surf be used? Championships couldn’t be rescheduled at very short notice and compensation would be due if an event was cancelled but in the end the surf turned up.
tonyb
TonyB makes a good point for the regular surf contests. However, the Big Wave contests (http://titansofmavericks.com/) which have wide windows and short callups may benefit from subseasonal predictions.
Howard
Excellent point.
AD hoc events would work but scheduled ones could cause problems if rescheduled unnecessarily due to a wrong forecast. Compensation looms!
tonyb
Howard,
The wild coast south of Pillar Point, which Mavericks is just offshore of, is my exploration playground. Because many of the wildest spots we explore are accessible only at extreme low tides, often with cliff climbing involved, my group uses NOAA buoy information about approaching swells just as the Mavericks organizers do. While a subseasonal forecast might be of use for some contests under certain conditions, knowing an offshore buoy is experiencing 25 foot waves headed your way in a couple of days is generally all we need to know.
I should mention that my brother has surfed for a long time and was in a surf related business for about 10 years. If the surf competition organizers had reliable forecasts of when large surf would be coming they would almost certainly change the dates to accommodate the large surf. For instance, the U.S. Open of surfing is held in Huntington Beach Calif in the summer when the surf is quite often poor. I am virtually certain that the organizers would move the contest a week or 2 later, if they were assured that they would have 10 foot waves instead of, let’s say, 3 foot waves.
JD
John:
Right now, the Mavericks contestants have a very narrow window using http://www.stormsurf.com/page2/services/about.html
which is more involved that checking a couple local buoys.
Maybe I’ll see you poke-polling out at Pigeon Point or Arroyo De Los Frijoles during the next king tide in a couple weeks.
http://california.kingtides.net/
On Anthony’s WUWT, Bob Tisdale is asking for predictions for El Niño conditions for 2015/2016. After reading his post, give your prediction and maybe why.
No El Nino.
‘Blow winds and crack your cheeks! rage! blow!
You cataracts and hurricanoes spout.
Till you have drench’d our steeples, drown’d the cocks!’
H/t the trade winds.
Simple. Just ask anyone who denominates himself a “climate scientist”. Something as large, important and close as an emerging El Nino or La Nina will be impossible for him to get wrong. The fact that nobody is sure if we are presently in an El Nino is immaterial. The past and present are always a bit tricky, with all that noisy info and opinion. So much comes down to terminology and degree when discussing stuff that has actually happened or is happening now.
But the future has such clarity! It is pure of polluting facts and events.
The fuchure so pure
of polluting facts.
I like that.Kinda’ like
Cli-sci itself.
I should have mentioned this earlier. Having grown up on skiis I was pretty aware of good years and bad. Short seasons vs long seasons and snow conditions are all important. I found this site for Mammoth:
“There are other sub-seasonal factors that will influence the winter season on smaller time scales that are not perdictive in the long term, but could have big impacts over short time frames and help shape the season as a whole”.
http://mammothsnowman.com/mammoth-mountain-weather/2013/10/18/official-2013-2014-winter-outlook-mammoth-mountain-california/
http://mammothsnowman.com/mammoth-mountain-weather/wp-content/uploads/2013/10/2013-10-18_0939.png
There would also be the importance of snow pack in the Rockies as far as drought. Also precipitation in general regarding drought and water use.
@ordvic
Interesting chart. I find the Seasonal–Forecast skill: Med-High, Forecast detail:above/below pretty interesting. I would love to see a mathematical development of why this is the case. Or even a numerical study that gives some support to this….
I also dig this chart because I believe it, since it is not the product of academia but the product of a group that has some skin in the game.
Nickels, They say they use previous years and models for the season. They also said this was a low confidence year because of a nuetral ENSO. They point to the years used.
I’ve been skiing and camping at Mammoth many times.
“to better understand and predict at the timescales of climate change, we need to work up ladder of the timescales, and figure out how to predict at the sub seasonal and seasonal time scales.”
Yes, yes and yes.
If there were actual $ to put to this research task it would be a pretty incredible problem. Actually tractable, at least to explore.
More modern solvers, aposteriori error analysis, innovative ensemble techniques.
More likely it will consist of throwing todays solvers together and figuring out how to view them as an ensemble, but whatever is learned will still be interesting.
So far these “short” term predictions have had a bad reputation because peoble remember the forecast when the weather appears. That is why 50 years forecast is so much easier.
On the other hand it is very simple to check if it has some value. Just put in the weather for june 2005, and check if it can predict the weather in september. Maybe that’s why it is in a limbo. Just ask Met office.
The need and use of such predictions is valuable, but they also need to be more accurate than pure guesses.
‘There is a new perspective of a continuum of prediction problems, with a blurring of the distinction between short-term predictions and long-term climate projections. At the heart of this new perspective is the realization that all climate system predictions, regardless of time scale, share common processes and mechanisms; moreover, interactions across time and space scales are fundamental to the climate system itself. Further, just as seasonal-to-interannual predictions start from an estimate of the state of the climate system, there is a growing realization that decadal and longer-term climate predictions could be initialized with estimates of the current observed state of the atmosphere, oceans, cryosphere, and land surface. Even though the prediction problem itself is seamless, the best practical approach to it may be described as unified: models aimed at different time scales and phenomena may have large commonality but place emphasis on different aspects of the system. The potential benefits of this commonality are significant and include improved predictions on all time scales and stronger collaboration and shared knowledge, infrastructure, and technical capabilities among those in the weather and climate prediction communities…
The global coupled atmosphere–ocean–land–cryosphere system exhibits a wide range of physical and dynamical phenomena with associated physical, biological, and chemical feedbacks that collectively result in a continuum of temporal and spatial variability. The traditional boundaries between weather and climate are, therefore, somewhat artificial. The large-scale climate, for instance, determines the environment for microscale (1 km or less) and mesoscale (from several kilometers to several hundred kilometers) processes that govern weather and local climate, and these small-scale processes likely have significant impacts on the evolution of the large-scale circulation.’ http://journals.ametsoc.org/doi/abs/10.1175/2009BAMS2752.1
https://watertechbyrie.files.wordpress.com/2014/06/hurrel.png
Doable in principle but it requires 1000’s of times more computing power, better data and more complete understanding of the system.
In the meantime we have inherently uncertain probabilistic calculations based mostly on sea surface temperatures. For instance – the IOD with effects on Australian and east African rainfall.
http://www.bom.gov.au/climate/IOD/indian_ocean-ssts.png
The biggest effects are in the Austral winter and spring as the Sun moves south through the tropical zone.
http://www.bom.gov.au/climate/IOD/negative/rg.NIOD.gif
A fun little system is the Madden-Julian Oscillation – the wet cell of which is currently over northern Australia adding to the monsoon.
http://envam1.env.uea.ac.uk/images/wh04_ppt1.gif
Understanding these sub-systems allows semi-quantitative predictions for up to a decade or so. It also builds understanding of the dynamics of the system on daily to millennial scales and the interactions between the system components on every scale.
Thanks for the link and for your thoughts on the prediction problem. The size of centralised computing power that would be needed would indeed be humungeous. Extension of the current synoptic capacity of 6-7 days to 14 days would be a great start, but the lack of sufficient weather in the oceans would make this difficult to achieve for vulnerable coastal communities.
err weather stations
It is just those sorts of increase in computing power that these guys are aiming at.
Satellites go a long to providing data across the depth of the atmosphere and the surface of the oceans. Instrument arrays in the oceans provide information on that domain. For instance –
https://watertechbyrie.files.wordpress.com/2014/06/26-degree-n-array.png
Putting it together builds a picture.
‘A principle that unites every kind of complexity theorist,and they are a richly varied class (see §3 and §5, below), is that observable ‘reality’ pertaining to any field, physics, biology, chemistry, applied mathematics, economics, etc., is complex but this complexity emanates from simple building blocks – of concepts, methods and rules of interaction. Why, then, should this supreme scientist, of powerful intuitions, claim that nature is the realisation of the simplest conceivable mathematical ideas? Is it because even the ‘simplest conceivable mathematical ideas’, when
realised in natural phenomena, become enveloped in complex manifestations and it is the task of the theorist to disentangle the apparent complexities and bare the hidden simplicities underpinned by, and in, simple laws and concepts?’ http://web.unitn.it/files/7_03_vela.pdf
With climate we are far from a certian resolution – des[ite all the overconfident guff generated.
Reblogged this on I Didn't Ask To Be a Blog.
Michael Mann (see linked article) writes, “Much as lions on the Serengeti seek out vulnerable zebras at the edge of a herd, special interests faced with adverse scientific evidence often… attack those researchers whose findings are inconvenient, rather than debate the findings themselves.”
And, they hide the decline, right?
Western academia’s radical interpretation of science by global warming climatists is more akin to radical ISIS than communist Russia. As members of the increasingly insular government-education establishment, the crippling anti-capitalism of the Left with its use of UN-approved mathematical models and CO2-fearmongering propaganda to justify its taking over of the economy is a high-tech lopping-off of heads.
How do I get elevated from sowing doubt among the socks to a top drawer delayer?
==================
…keep darning ’til the thread breaks?
Get naughty with lazy tattling? Naw, knotty with laced tatting.
======================
Dang, ‘lacy tatting’.
========
I thought I was going to read a science paper but it was an editorial. It reads like comedy, It sounded like one of FOMD’s posts. He used all the cliches including 97%, Rachel Carlson, big oil, big tobacco, Exxon, Kock brothers, swift boat, Potemkin Village, creationism, industry funded front groups and hired guns, anti-science, flat earth, orchestrated attacks, etc etc. It sounded like a comedy routine. He ended by calling himself a distinguised professor and part of the IPCC Nobel prize. Quotes from Carl Sagen and Mark Twain never hurts. None of this is new and is still just as laughable. Maybe that was the point. I have to credit him, he was original on a couple of points. The Deniers seem to have given up and have become Delayers. One good dart, very smart. The serengeti analogy may not be such a good idea, after all the lions attach the weaklings, the less experience or less intelligent of the herd. I think the Michael the Archangel works much better for the poor, maligned, martyr of climate science.
Once you factor in all of the excuses the long term prognostications of the official global warming establishment have been spot on. And, just talking about how fearsome the problem of AGW is, much has been accomplished –e.g., irrespective of exactly what made it happen (we have a Eurocommie-approved Nobel Laureate who takes credit for it, which also is the consensus of opinion among Leftists), we apparently can all take it as given that the seas were rising but now they’ve stopped, much like the globe was warming but now… it’s stopped.
My company has a need for such forecasts. We specialize in producing street parking data sets, and having medium range forecasts allow us to plan much more efficiently on how/when to schedule between different cities. Rain, for example, is very bad for our equipment.
Thx for this one
OT: according to Michael Mann’s latest demented screed, Judith Curry now belongs in the category of the evil “delayer” — not an outright “denier” but one who seeks to delay urgently needed actions:
http://bos.sagepub.com/content/71/1/33.full.pdf
I note that Mann tries to paint virtually all criticism of his work as politically motivated and does not even address any substantive statistical and scientific issues.
Geniuses like Dr. Mann know it all. I am not sure how; they might be hearing voices. Unfortunately voices are silent when it comes to sub seasonal forecasts (or shorter).
‘Serengeti Strategy’ * nice ass-onance here. Some well
employed spondees in the article, ‘fight-back,’ ‘out-reach,’
bad-faith,’ ‘ well-heeled,’ and finely-tuned dactyl descriptions,
‘doubt-sowing’, ‘surf-boating,’ (serf’s like that) ‘quick-recap,’
and ‘delayer.’
* Sereneti Strategy is necessary when there’s widespread,
even as high as a 97% consensus of scientists knowing
that climate-change is real. Say, whatever happened to
the former terminology, ‘mann-made global warming?’
I think you already know this but for what it is worth.
Based on what happened to generating companies during last year’s polar outbreaks, a several week forecast that extremely cold weather was a high probability would have been useful. When it is very cold natural gas used for generation is diverted to heating and dual-fueled units switch to oil. Ten years ago the differential between oil and gas was close enough that oil was economic several times a year. In the last ten years the differential has so advantaged gas that economic oil runs have become very rare. Consequently, there has been little incentive to purchase oil and last year the stocks were so low that there was a lot of scrambling around to get oil during the really cold weather. A reliable several week forecast could have been used to forestall the crisis mode and probably would have enabled users to provide oil fired power profitably.
In the summer if we get really really dependent on wind and you can forecast that a monster high pressure system is going to stall over the eastern US, that also will be valuable information for the generating companies and the system operators.
An economist once told me that in making economic forecasts, you have to give them numbers and dates, but beware of tying them together.
Here in Australia, the CSIRO has identified one of the world’s best climates. (The fishing’s not bad either.)
http://news.domain.com.au/domain/real-estate-news/yambas-best-beach-getaways-20150107-12ivjm.html
This ‘best climate’ was not predicted anywhere.
By anyone.
“Whatever the reason, the punters keep returning year after year and some eventually make it their permanent sea change.”
Gotta be the world’s best kept secret.
Perfect climate, year after year, while the rest of the world faces hottest year ever as we face our co2 induced bush fire burning hell.
Yes, I am mocking the 97% certified settled climate science.
FUTURE CLIMATE
Those who are interested in learning why long-term (~500 year) cooling will start after the year 2059 can see my new website whyitsnotco2.com.
I assume that you plan to retire in 2059 after a long successful career. (That’s how warmists operate.) Please try make a prediction for my time frame, I have a very promising future in my past.
Well, there are a number of people predicting cooler sooner. Not sure that the 2014 prediction was wrong as much as a few years premature. We’ll see.
http://www.esrl.noaa.gov/gmd/ccgg/trends/
December 2014: 398.78 ppm
December 2013: 396.81 ppm
NOAA says the CO2 level went up 2.32 PPM in 2014. Given the actual numbers they must be using an “adjusted” CO2 level or new math.
With the atmospheric CO2 increase showing signs of starting to taper off, I’m not sure that it will get much warmer. But I’m not in one of the cooling camps quite yet. The CO2 level could top out before 2059 which would help your prediction.
Yes, but there will be 30 years of warming between 2028 and 2058 – see the calculated plot at whyitsnotco2.com based on planetary orbits.
I like your chart. As I read the chart you show enough warming between 2028 and 2058 to prevent mass seppuku by the warmists but not enough warming to save the models.
We know the models (thus the current consensus theory) is wrong. Your theory is interesting. You seem to think there are strong direct and indirect solar influences. I’m pretty sure the solar influence is greater than that assumed by the consensus so I’m at least half-way in your camp.
No PA – I have explained the valid physics which rules out all potential warming by carbon dioxide, water vapor, methane etc. If you (or any reader) thinks there is some valid process then describe it and I’ll explain what’s wrong in that explanation.
The climate cycles correlate exceptionally well with the 934-year and superimposed 60-year cycles in the inverted plot of the scalar sum of the angular momentum of the Sun and all the planets. The correlation is so compelling (as you can see at a glance) that we can be confident that it will continue in future centuries. I first pointed this out over three years ago on my first climate website for which the URL is shown in the lower right corner of that plot here and you can read the breakthrough physics I’m discussing on that page.
This has the potential to be world shattering, but it may take 10 or 15 years to be recognized However, those who are astute in their understanding of physics, especially the Second Law of Thermodynamics, could play a part in disseminating this information pertaining to the gravitationally induced temperature gradient that is found in all planetary tropospheres and sub-surface regions and the resulting energy flows.
.
PA:
Thanks for that. The climate cycles correlate exceptionally well with the 934-year and superimposed 60-year cycles in the inverted plot of the scalar sum of the angular momentum of the Sun and all the planets. The correlation is so compelling that we can be confident that it will continue in future centuries. Ttat plot is here and you can read the breakthrough physics I’m discussing on that page.
I did that over three years ago. I’ll copy it below … I probably got the 2014 wrong – maybe this year instead.
“From 2003 the effect of El Niño had passed and a slightly declining trend has been observed. This is the net effect of the 60-year cycle starting to decline whilst the 934 year cycle is still rising. By 2014 the decline should be steeper and continue until at least 2027. (This statement was archived 22 August 2011 here)
My solar based long range forecast for 2014 included deterministic forecasts for negative NAO/AO and US Arctic outbreaks from around 7 Jan (for ~2 months), 10/11 Nov (for ~2 weeks), 26/27 Dec (for ~3 weeks).
Detail for the UK included deterministic forecasts for heatwave conditions from ~22/23 July, cooler and wetter through the middle third of August, and a very warm signal from ~2 Sept lasting most of the month.
Given that hindcasts giving such detail can be made back several hundred years from the heliocentric planetary ordering of the daily-weekly solar signal, such NAO/AO based forecasts can be produced for any useful range.
The next period for stronger Arctic outbreaks, I have from around 18th March 2015.
Some commentators @climate.etc claim humans are carrying out an experiment on the planet & atmosphere.
The Scientific Method
Theory/experiment:
1990: Finland introduces the world’s first carbon tax to stop “Doomsday Global Warming.”
http://www.sbs.com.au/news/article/2013/10/29/factbox-carbon-taxes-around-world
2015: Finland’s temperatures have risen at roughly double the rate of the planet as a whole, a new study suggests
http://www.scientificamerican.com/article/land-of-the-midnight-sun-warms-fastest-in-world/
If observations disagree with experiment, it is wrong.
https://www.youtube.com/watch?v=EYPapE-3FRw
How many more years experiment are needed before a conclusion is reached?
http://s24.postimg.org/6ztf70ql1/Helsinki_Temperatures_1829.png
Making the case for extreme warming with the raw data is a little hard.
There must paper out there that says temperature data is like canned milk but a google of “surface temperature” and “canned milk” doesn’t turn up anything.relevant.
Anyway – if you homogenize, pasteurize, sweeten, and condense the temperature data from Finland it show significant warming.
I guess it comes down to whether you believe temperature data is a dairy product or not.
Greetings PA. Thanks for your reply & time. If I may Quote you:
“Making the case for extreme warming with the raw data is a little hard.”
Quite so.
6.1.2015:
“Weatherwise, 2014 was an extraordinary period in many ways.
According to the Finnish Meteorological Institute FMI, last year will go down as the second warmest in Finnish history.
According to the weather experts at FMI the only other year to beat the average heat record in Finland in 2014, was 1938 …”
Does anyone here have a graph of carbon(sic) levels in 1938?
Wait! http://www.esrl.noaa.gov/gmd/ccgg/trends/weekly.html
Better check that.
25 years of carbon(sic) tax.
Where is the evidence the tax stops doomsday climate change?
But, there is more!
Finland beaches keep getting larger due to falling sea level…
Permanent Service For Sea Level: Helsinki
http://www.psmsl.org/data/obtaining/stations/14.php
Oops!
Missing link:
2014 the second hottest year in Finnish history
http://www.adn.com/article/20150106/2014-second-hottest-year-finnish-history
Mark M you’re welcome….
The articles appear to be accurate and Finland is warming. The headlines imply it is recent, the article says it has been going on for 200+ years. The CO2 level depending on who you want to believe was 290-310 PPM.
So 90-110 PPM of CO2 has brought us back to 1936-1938 temperatures.
Quoting from one of the articles:
“A team of researchers from the University of Eastern Finland and the Finnish Meteorological Society found that over the past 166 years, the country’s average monthly temperatures have increased by more than 2 degrees Celsius (3.6 degrees Fahrenheit), a 0.14 C change per decade.”
Now if it just as warm today as 1936 there were 2°C of warming by 1938, this creates some problems with global warming theory.
http://www.ipcc.ch/graphics/ar4-wg1/jpg/fig-3-1.jpg
http://hadobs.metoffice.com/hadcrut3/diagnostics/compare_datasets_hadsst3_logo.png
Two things jump out:
1. The articles compared the global temperatures to the Finnish (Finish) land temperatures.
2. Either the sinking of the Titanic or WWI significantly lowered global ocean temperatures. The assassination of Archduke Franz Ferdinand at Sarajevo is assumed to have no impact on global temperature due to the thermal mass involved and distance from the ocean.
http://hadobs.metoffice.com/hadcrut3/diagnostics/compare_datasets_crutem4_logo.png
The correct comparison was CRUTEM4 land temperatures. I used google maps to find Helsinki and it is on land so I assume most of the Finnish land temperature measurements were taken on land.. The correct comparison is 2°C/1.2°C or about 67% warmer. The article was either bad at math or used the wrong temperature set.
Statistically, small areas have much larger annual oscillations than the global average, so this type of variation would be expected. We are not setting global records everywhere every year because of this, even though 2014 is one of the warmest on average. I thought this was well known, but apparently it bears repeating for the newbies.
>… 2014 is one of the warmest on average
Compared to when, Jimmy Doo ?
I thought comparisons required specific time periods and this wold be well known, but apparently it bears repeating for dills
…compared to when records began.
even though 2014 is one of the warmest on average
Where?,the US for example was the 34th warmest year.
globally
“globally”
Where the magic happens. There’s never been a better word for an imaginary way to go to avoid reality.
Andrew
‘Finally, the presence of vigorous climate variability presents significant challenges to near-term climate prediction (25, 26), leaving open the possibility of steady or even declining global mean surface temperatures over the next several decades that could present a significant empirical obstacle to the implementation of policies directed at reducing greenhouse gas emissions (27). However, global warming could likewise suddenly and without any ostensive cause accelerate due to internal variability. To paraphrase C. S. Lewis, the climate system appears wild, and may continue to hold many surprises if pressed.’ http://www.pnas.org/content/106/38/16120.full
Finnish temps are not at all surprising – but the ‘cycles’ are shifts that are are not at all predictable.
https://watertechbyrie.files.wordpress.com/2014/06/finland-temps.png
http://download.springer.com/static/pdf/96/art%253A10.1007%252Fs00477-014-0992-2.pdf?auth66=1420697802_335a49996efe6602771168a89e305795&ext=.pdf
The linked Spiegel article (Spiegel Dumps Cold Water On “Record Warm Year” Significance … Sees Science Fraught With Widespread Uncertainty) can be boiled down –e.g.,
A UN panel says serious warming will return someday and when it does it will be worse than ever before if the emission of greenhouse gases are not dramatically curbed now. How do we know? That is what our mathematical programs tell us and there are many of these programs and they all tell us the same thing: in the long-term, we will be right no matter what anyone else may believe and despite what we may currently observe now or in the immediate future.
The real problem is that the global warming alarmists of the Left have no uncertainty whatsoever about their belief in Marxism — despite the fact that in practice it has been a socio-economic failure and the cause of immense suffering and millions of deaths — and, it is that certainly that wipes away their uncertainty about all else.
The most important point in the passages you quoted were the ones about “knowing when to fold ’em.” I would have preferred a blackjack card-counting analogy but the idea is similar: Given the apparently very tall technical challenge involved the key may be to identify those times and places when the cards are in your favor and outcomes are unusually predictable. Unfortunately, if such situations occur infrequently the only customers are likely to be financial speculators who can choose when to invest and way to lay back.
If you were somehow able to generate enough predictability, I would think that the construction and fishing industries would be eager customers.
One of the best I have run across. I have to get the most recent edition.
Kepner, Charles H., and Benjamin B. Tregoe. “The Rational Manager: A Systematic Approach to Problem Solving and Decision-Making by Charles H. Kepner, Benjamin B. Tregoe: McGraw-Hill Book Company 9780070341753 1st – Better World Books.” http://www.abebooks.com/Rational-Manager-Systematic-Approach-Problem-Solving/8277049288/bd.
Kepner, Charles Higgins, and Benjamin B. Tregoe. The New Rational Manager: An Updated Edition for a New World. Princeton, N.J.: Princeton Research Press, 2013.
Under “Situational Awareness” and “Problem Analysis” I suggest considering Alternative Possible Causes. One might avoid “barking up the wrong tree”.