by Judith Curry
In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not.
False Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
JP Simmons, LD Nelson, U. Simonsjohn
Abstract. In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.
Psychological Science published online 17 October 2011 DOI: 10.1177/0956797611417632 [full text]
Neurologica blog
The paper is discussed by Steven Novella at neurologica blog Publishing False Positives. Some excerpts (JC bold for emphasis):
In their paper Simmons et al describe in detail what skeptical scientists have known and been saying for years, and what other research has also demonstrated, that researcher bias can have a profound influence on the outcome of a study. They are looking specifically at how data is collected and analyzed and showing that the choices the researcher make can influence the outcome. They referred to these choices as “researcher degrees of freedom;” choices, for example, about which variables to include, when to stop collecting data, which comparisons to make, and which statistical analyses to use.
Each of these choices may be innocent and reasonable, and the researchers can easily justify the choices they make. But when added together these degrees of freedom allow for researchers to extract statistical significance out of almost any data set. Simmons and his colleagues, in fact, found that using four common decisions about data (using two dependent variables, adding 10 more observations, controlling for gender, or dropping a condition from the test) would allow for false positive statistical significance at the p<0.05 level 60% of the time, and p<0.01 level 21% of the time.
This means that any paper published with a statistical significance of p<0.05 could be more likely to be a false positive than true positive.
Worse – this effect is not really researcher fraud. In most cases researchers could be honestly making necessary choices about data collection and analysis, and they could really believe they are making the correct choices, or at least reasonable choices. But their bias will influence those choices in ways that researchers may not be aware of. Further, researchers may simply be using the techniques that “work” – meaning they give the results the researcher wants.
Worse still – it is not necessary to disclose the information necessary to detect the effect of these choices on the outcome. All of these choices about the data can be excluded from the published study. There is therefore no way for a reviewer or reader of the article to know all the “degrees of freedom” the researchers had, what analyses they tried and rejected, how they decided when to stop collecting data, etc.
They hit the nail on the head when they write that the goal of science is to “discover and disseminate truth.” We want to find out what is really true, not just verify our biases and desires. That is the skeptical outlook, and it is why we are so critical of papers purporting to demonstrate highly implausible claims with flimsy data. We require high levels of statistical significance, reasonable effect sizes, transparency in the data and statistical methods, and independent replication before we would conclude that a new phenomenon is likely to be true. This is the reasonable position, historically justified, in my opinion, because of the many false positives that were prematurely accepted in the past (and continue to be today).
Requirements for authors and guidelines for reviewers
The paper makes the following recommendations:
Authors:
- Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.
- Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data- collection justification.
- Authors must list all variables collected in a study.
- Authors must report all experimental conditions, including failed manipulations.
- If observations are eliminated, authors must also report what the statistical results are if those observations are included.
- If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate.
Reviewers:
- Reviewers should ensure that authors follow the requirements.
- Reviewers should be more tolerant of imperfections in results.
- Reviewers should require authors to demonstrate that their results do not hinge on arbitrary analytic decisions.
- If justifications of data collection or analysis are not compelling, reviewers should require the authors to conduct an exact replication.
They also discuss other options that they feel would not be effective or practical. Disclosing all the raw data is certainly a good idea, but readers are unlikely to analyze the raw data on their own. They also don’t like replacing p-value analysis with a Bayesian analysis because they feel this would just increase the degrees of freedom. I am not sure I agree with them there – for example, they argue that a Bayesian analysis requires judgments about the prior probability, but it doesn’t. You can simply calculate the change in prior probability from the new data (essentially what a Bayesian approach is), without deciding what the prior probability was. It seems to me that Bayesian vs p-value both have the same problems of bias, so I agree it’s not a solution but I don’t feel it would be worse.
JC conclusion: While the examples and recommendations are specific to the field of psychology, these general issues are germane to any scientific problem that uses statistical analysis and inference. Lets look at the hockeystick papers (MBH 98, 99) as an example. How should these rules be modified for such a study? Based upon what we now know about these papers and the background of what went into them, how do these papers measure up by the standards of the false positive analysis?
What are the odds?
=====
How do you feel about the allegation that you are Mosher’s bot?
Does he speak ‘through’ you?
You write like Joshua.
And nobody else would care.
Hmmm.
I’m not sure anybody would want to speak through Joshua’s bot ;)
Bot-check.
you obviously have not read the back and forth between the mosh bot and the kim bot. it goes back 4 years at least.
its a grand experiment, sock puppet
Wow! Some folks in these here parts really are obsessed with me, aren’t they?
“You write like Joshua”….hmmm, for some, them would be fightin’ words
No doubt. Particularly for those who fantasize about me, dreaming up that the can see “windows into [my] soul,” or dreaming about my posting comments as a sock puppet.
All this fantasizing about me would be flattering if it weren’t so creepy.
Prob. Spring of ’07 @ Stevie Mac’s.
==========
Josh speaks out of his own bot.
Joshua.
Read more carefully.
bot-check is a sock puppet that writes like Joshua.
dear god are you dense or what
First we have this:
And then we have this:
Too funny.
Joshua,
you still dont get the joke. sorry its an inside game.
steve mosher: bot-check is a sock puppet that writes like Joshua.
Good grief! Is everyone here a bot?
MattStat –
Do you mean you’re not a bot?
What’s it like?
According to the scientists, they are better than even or so it seems.
They love the soap.
Good grief! The article is important. What is this tangent?
O.K., not that surprising but it has nothing to do with the kinds of incentives and social bias found among AGW advocacy science. Nuclear bombs vs. handgrenades.
Zealot AGW science for a cause doesn’t realize their bias? That’s a tough sell, they know and it’s rationalized;
“Each of these choices may be innocent and reasonable, and the researchers can easily justify the choices they make. But when added together these degrees of freedom allow for researchers to extract statistical significance out of almost any data set.”
It’s way more than that in the case of AGW and the world knows it. This is minimizing what has happened morally and in the greater scale of historic AGW advocacy science.
“In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not.”
Raymond Davis Jr. reported evidence that the expected number of solar neutrinos did not exist for many decades until he finally received a Nobel Prize in 2002.
In his autobiography, he says, “My opinion in the early years was that something was wrong with the standard solar model” although “many physicists thought there was something wrong with my experiment.”
http://www.nobelprize.org/nobel_prizes/physics/laureates/2002/davis-autobio.html#
Dr. Raymond Davis, Jr. was the oldest person to receive a Nobel Prize in Physics.
Thank you Oliver. I didn’t know that. :)
Dr. Andrew Davis, his son, delivered his father’s Nobel Prize Lecture.
Although “spin artists” tried to present his work as confirmation that solar neutrinos oscillate, in fact, the measurements that supposedly confirmed the standard solar model were published by 178 scientists in 2001, a few months after that problem was solved and reported at the Lunar Science Conference in March 2001.
Climate science is not about bleeping anecdotes.
Call me a sour puss (I really don’t care), but unless every assertion that people make is accompanied by some statistical measure of its potential universality, it’s irrelevant noise.
I know the game. Anecdotes are good if they support your position, but are suspect when contrary.
The theme of this thread is that there are four possibilities for any conjecture:
1. The false positive
2. The false negative
3. The true positive
4. The true negative
Anecdotes can fit in anywhere in this categorization, but unless more supporting evidence is available, the anecdote is a Bayesian update, useful only if given a proper prior weighting.
The case of Dr. Raymond Davis does not prove anything.
See how this works?
Thanks for your information. The security of society depends on real knowledge. Leaders of nations and sciences spent vast quantities of public funds to these obtain Inconvenient Truths:
1. Truth #1 is in this brief (3-min) video of images from space:
http://www.youtube.com/watch?v=fKTu6B4Rgek
2. Truth #2 is in this summary of space age data:
Earth is a tiny piece of fly ash heated by the
Nuclear furnace that made our elements and
Spit out the ash five billion years (5 Gyr) ago
3. The key to Truth #3 is in this 3-D plot of rest masses of atoms:
http://www.omatumr.com/Data/2000Data.htm
Truth #3 is written in the “Cradle of the Nuclides,” published in 2000 on the cover of Proceedings of the 1999 ACS Symposium organized by Glenn Seaborg and me to show how nuclear energy is stored as rest mass in mixtures [of two forms of one basic particle (Neutron/H-atom)] that comprise every atom:
http://www.amazon.com/Origin-Elements-Solar-System-Implications/dp/0306465620
Deception also blocked a solution to our energy needs:
http://bravenewclimate.com/2012/01/08/nuclear-fission-flyer/#comment-148404
Thanks to a few brave souls – Jeff Id, Steve McIntyre, Prof. Judith Curry -we have a channel to communicate factual information that accumulated behind a wall of government deceit before surfacing as Climategate email and documents in late Nov 2009.
With kind regards,
Oliver K. Manuel
Former NASA Principal
Investigator for NASA
The placebo effect applies to data, too.
Maybe the fun folks of CAGW just need bigger pools…
http://www.washingtonpost.com/blogs/federal-eye/post/congressional-wildfire-office-pool-angers-firefighters/2012/01/11/gIQA3unrrP_blog.html
Why knot, a mosh pit. You still can dance up a sweat and not feel dirty.
I am reminded of the Church and White sea level study and the tide gauge data that they eliminated or truncated. There are over 1200 tide gauges in the PSMSL and less than 650 were used.
How about for a multiproxy paleoclimate study:
1) No new methods in same paper as new compilation.
2) Must use all previously published original data sources in full (not previous multiproxies) or provide explicit reasoning why not.
It strikes me that BEST is pretty close to fulfilling most of the conditions here. But that type of analysis (and multiproxies) have additional challenges because they are meta-analyses drawing on archival data.
Looks to me BEST comes up way short in A4, A5, A6. The scalpel is all about omitting data. Where is the analysis that the scalpel and suture doesn’t bias the conclusions? Arguments that the scalpel does bias against low frequency data are at least nine months old.
BEST ?
Ignore the oceans and pretend that the whole planet is land ?
Hmmm…..
They are working on the oceans right now. Soem think it will take more than a year. I’m it’s less than a year.
Doesn’t matter. All you have to do to see the the BEST result is look at GiisTemp!
I have a slight disagreement with the emphasis on ‘false positives’, which may be because of the subject matter – climate science is a field less concerned than psychology with the dichotomous true/false, yes/no, proven/not proven paradigm. It is surely more concerned with how much?, when?, for how long?, with what sort of impact?
Having said that, the psychological points are very pertinent. My question about MBH would be about how honestly the available data is represented by the final graph. I say ‘honestly’ cautiously – I don’t mean to imply fraud, or deliberate/conscious bias in the data gathering/analysis. But the (powerful) psychological biases hinted at in the article can easily – and obviously – lead to the data not being represented honestly. In fact it can – and maybe did – lead to the data ‘saying something’ that it would confess under only the most severe torture.
And then, of course, any ‘independent’ verifications made by friends, close colleagues and those who have already publicly made their views known, are even more subject to the psychological pressures that miss-represented the data in the first place.
The experiments subsequent to Milikan’s oil drop experiment are a perfect example of this. Armed only with a respect for a well-known physicist, many independent researchers repeatedly found ways – unwittingly – of dishonestly representing the data. They weren’t dishonest themselves, but their conclusions (their measurments, even) again and again failed to follow from the evidence in front of their eyes.
It is not only – or even primarily – because of deliberate fraud that medical trials are double-blind. We see, choose, and interpret mostly what we want to see – whether we like it or not and whether we realise it or not. It doesn’t help that some people are more objective (better at keeping biases at bay) than others – we should be suspicious of all and sundry. James Hansen and Richard Lindzen have pretty much the same information before them, but is anybody holding their breath waiting for their position on climate sensitivity to be reversed?
The most striking sentence of the article for me – and one that transfers easily from psychology to climate science – is this
“But when added together these degrees of freedom allow for researchers to extract statistical significance out of almost any data set”
Think how many degrees of freedom MBH had in their reconstruction? How many choices of data, proxy, weighting, interpretation, analytical tool etc etc. To me, the one single crucial question to be asked when assessing the likely ‘honesty’ of the study is this – did the researchers have a slightly desired outcome before they started making their innumerable choices? I’ll stick my neck out with Michael Mann at least, and say that it is my impression that in his ego-mania and hubris, he’d probably dreamed of the shape of the final graph a thousand times before making any of those rather arbitrary choices. The climate world was positively begging for a new hero who could simultaneously provide a devastating graphic at the same time as both screaming ‘unprecedented’ and ‘MWP? – what MWP?’
I’ll end with something that I find very striking – in its ordinariness. On the first entry of Chris Colose’s blog, when he was an undergraduate a few days out of his teens he makes this statement –
“Scientists are clear on a few issues- the globe is warming, and we’re responsible. Climate Sensitivity is enough to be worried. It will take effort, and money, to solve the problem but the benefits outweigh the risks.”
I pick this out because my experience of Chris’s writing leads me to believe that he’s one of the least partisan, most honest AGW scientists around. My point is that a world view was already established such that
there was enormous hidden pressures to make choices – all those that MBH made, plus dozens of others concerning subject matter, job applications, choice of colleague etc. I would expect the majority of climate scientists to be even more susceptible to unwitting bias than Chris.
A combination of these pressures, and the ability of climate scientists (because of the nature of the subject) to be able to find any statistical significance out of almost any data set” is what informs my climate ‘scepticism’. That, and the observation that some people have been saying (erroneously) that ‘we’re heading for a catastrophe!’ every day since man first learned to speak.
In a true scientific sense, we should be concerned with ranges of uncertainties, and P10-P50-P90 ranges on uncertain quantities.
But in the politically funded, politically motivated, post-normal science, precautionary principle driven behavior to “act now before it is too late”, we are very much in the “probability that premise A is TRUE”, “science is settled” mode.
There needs to be one or more reviewer rules that subject claims of Precautionary Principle (PP) to additional scrutiny. This is not to so much ban the PP from use, but by its very definition it is an appeal to short circut scientific rigor, so a higher level of methodologic scrutiny seems needed in the peer review.
Judith Re: How should these rules be modified for such a study? How do these papers measure up by the standards of the false positive analysis?
In The Impact of Yamal on the Spaghetti Graph Sept. 29, 2009, Steve McIntyre observed:
To extend the “rules” to “climate science”:
1) All statistical methods used must first be objectively validated in peer reviewed literature.
2) All available applicable data must be used without cherrypicking for desired results.
3) Methods known to give the appearance of desired results from random noise must not be used.
PS Original documentation available at ClimateAudit.org under Hockey Stick Studies
See Steve McIntyre below and at ClimateAudit.org Summarizing:
4) Demonstrate method reliability “out of sample”. (cf “Yamal”)
5) Exclude types of data known to give poor results, especially when blue ribbon panels warn on avoiding them. e.g. bristlecones, e.g., “The NAS Panel found in one place that “strip-bark samples” (which Graybill sought out in his bristlecone collections) should “not be used”.”
6) Publish the data right side up! (cf “the bizarre handling of the Tiljander sediments in literature cited on multiple occasions by IPCC AR5″)
7) Correct the scientific record when papers are known to be in error.
See: Red Wine Researcher Said to Falsify Data
“They referred to these choices as “researcher degrees of freedom;” choices, for example, about which variables to include, when to stop collecting data, which comparisons to make, and which statistical analyses to use”
1. as mcintyre argues the proxies need to be updated.
2. Nobody has seen the reconstructions with/without the decline
thats just a start.
one reason we ask for code is so that we can investigate this VERY QUESTION. When people argue that reproduceability is less important than independent replication, they forget this aspect of the problem.
To investigate researcher DOF, your best approach is to use the code that was used for the primary research. Mann’s code with and without BCP, with and without tiljander, with and without … you get the idea.
Over at CA I’ve called these uncertainties ” uncertainty due to analyst choice”, but I think researcher DOF is probably more apt.
I also like the term “research degrees of freedom”, this is exactly what needs to be understood.
Isn’t climate science as a whole relatively drenched in them? Isn’t that what makes engineers etc goggle at the vast number of choices/assumptioins that have to be made? And thus the inappropriate degree of ‘certainty’ that drips out of the other end of the process?
I wonder if a comparison could be made between various sciences on just this issue. Are such comparisons possible?
Yes, it is difficult to have a high level of confidence/certainty in such analyses with so many degrees of freedom for the researcher.
Anteros,
I think your point is essential, but it applies of course to many fields of research, where data is scarce, fragmentary and obtaining better data either impossible or very slow. That should be accepted without goggling, but the scientists should be fully open on the effects of the data limitations for their conclusions.
Very much scientific progress has been obtained studying rather unsatisfactory data. Such research doesn’t lead directly to firm conclusions but rather to hypotheses to be confirmed, disconfirmed or refined by further research. In that kind of science the methods of statistical analysis are often non-orthodox, as that may help in finding the signal in noisy data, but that means also that proper error analysis may be impossible and that the uncertainties can be estimated only in very general terms.
Yes Anteros,
from my experience I would agree to some form of that. Also, marketeers of dish soap and enemas do more parametric variations on their assumptions and choices than any paleo work I’ve read.
pardon that image.
pekka –
I agree with what you say, and think that the problem arises when either scientists or the public are led to believe that the degrees of freedom are less than they really are. This pertains to communication and the prime example of Al Gore suggesting that climate science is a hundred times more certain than reality warrants – it has ‘spoken and is ‘settled’. In fact all advocacy does this and the more cautious voices are inevitably lost in the background. It is not only the ‘frightening’ that sells but the definitive. And a study that merely ‘suggests, tentatively’ is often sold [and even more importantly, bought] as cast in stone.
Witness the number of people that have ‘bought’ the tentative suggestion (reached with any number of degrees of freedom) that 50% of all species will be extinct by mid-century.
I have some sympathy with those who are already persuaded that an apocalypse is afoot, in that patient, cautious teasing out of the signal from the noise is hopeless if urgency is called for. There are thus many thousands of people going out of their way to show what they already believe and what chance has objectivity is such circumstances? I think the problem is very much wider and deeper than the headline ‘noble cause corruption’; it is a situation where the appropriate science is always going to struggle to match the subtly motivated and ‘confirming’ variety.
I wonder if a comparison could be made between various sciences on just this issue. Are such comparisons possible?
That’s an interesting point, Anteros. As someone familiar with more than one branch of science, I hesitate to predict. However, I see the issue as having two separate aspects. The first is the frequency of false positives, and the second is their durability. Although I don’t know how various fields of science would compare regarding the first, there are some principles that should affect the second. Science staggers toward “truth” through a trial and error process that sifts out the false from the true via the essential ingredient of replicability. For an important result to be accepted, it must be reproducible in the hands of others. That isn’t enough, however, because if the result is universally accepted without challenge, the same propensities that generated the false positive in the first place will tend to reproduce it. On the other hand, I expect that important false positives will rarely prove to be durable in a field that is constantly subjected to challenge, particularly if many of the challengers are strongly motivated to demonstrate the falsity.
Ironically, in very contentious fields, I wouldn’t be surprised to find that false positives are disproportionately frequent, and disproportionately short-lived, although during the process, it may not clear which positives are false and which are the subject of false claims that they are false.
Anteros,
I have written many times that I consider risk aversion to be justified and important (to avoid the words “precautionary principle”). That gives much weight to uncertain estimates and large risks even, when their probability is rather low, while not negligible.
What I dislike very much is the attitude that people don’t understand the correct arguments and therefore we must lie. Sometimes this may work, but very often it backfires. Even, if I agree that the risk is big enough to warrant action and if lying works, the problem remains that the possibility of analyzing the right level action is lost. Without that good decisions are possible only with best of luck.
Pekka, “What I dislike very much is the attitude that people don’t understand the correct arguments and therefore we must lie. ” 8O
When Michele asks if her shorts make her butt look fat, I should lie, I may lie, I consider lying but I am not required to lie.
The other side of “risk” aversion is to
1) avoid the risks of alarmist researcher bias
2) avoid descending into tyranny
Fred,
I would say that in a contentious field where there are political implications that false positives tend to last longer because the scientists have stronger motivation to not admit error.
I think this problem is particularly prevalent in fields where the data is noisy and there are strong feelings involved. I would say that medicine is such a field, but that’s an art and not really a science. There are numerous examples that lasted for a long time such as vertebraeplasty.
And there are I think more self imposed standards in medicine, like double blind studies and statisticians on the team from the beginning. But still, a huge amount of money is wasted on treatments that have no statistically significant benefits. People have very strong feelings about their own bodies and their health and so its hard to bring rationality to medicine. For a lot of doctors, the path of least resistance is to “do something” to make the patient feel that there is hope, even if the something doesn’t really help very much.
You and I have had this argument before, but I think the evidence is very strong that climate science is particularly subject to false positives compared to more mature fields like fluid dynamics. So, we are probably not going to agree now, but it seems to me that the whole paleoclimate Mann controversy is a clear case of a false positive. I really respect McIntyre much more than Mann if for no other reason than that his public statements are far more calm and rational. And I see the climategate emails show that even members of the team in private agreed with McIntyre. Others issues include the models and their seeming centrality to the field and all that that implies. Another is the “adjustments” of the data regarding the tropospheric temperature trends with altitude. The other problem is the quality of the data for such a complex and large system.
It is more politically correct that “making stuff up” but the end result is pretty much the same. I do like having a new PC pejorative, though.
In the Harley community I associate with DOF expands to “decrepit old [f-word of your choice]” and which still seems appropriate to much that appears in climate research.
When reading this paper originally, the aspect that struck me with respect to researcher bias and Dendrochronology didn’t have as much to do with later stage processing and analysis of the data as it did with the very early stages of interpreting the data and deciding what individual trees and tree rings actually meant and whether to include them or not as a proxy. The issue came to mind due tou a discussion of one of the Climategate 2.0 emails that was discussed fairly recently at WUWT. The WUWT post is less interesting in relation to this (IMO) than the original email. http://www.ecowho.com/foia.php?file=1738.txt
The email chain highlights a tree physiologists concerns with what individual tree rings actually mean and how the Dendrochronologists “select” what data they choose to include/exclude. The discussion appeared to scream researcher bias, both unconscious and very likely conscious bias
Paleo Climate Scientists start with the assumption that Milankovitch Cycles and CO2 and Solar Cycles are responsible for Major Warming and Cooling to produce Ice Ages and Major Warm periods and do not mention any possibility of anything else being responsible. They don’t even look for the possibility that they could be wrong. A Scientist who is not Skeptical is not a proper Scientist. Every good Scientist is a Skeptical Scientist. Consensus Science is not real Science.
They do not consider the possibility that Maurice Ewing and William Donn were really correct and Ice Albedo is the controlling forcing that Earth uses to control the Thermostat of Earth. Soon, the current cycles of Low Arctic Sea Ice Extents will bring down enough snow to cause Climate Theory to be reconsidered.
Ewing and Donn were right. Wysmuller and Pope are right. Leighton Steward is right about CO2. More is better and it is not going to cause any major warming.
When the Arctic is open, it snows, ice advances and Earth Cools. When the Arctic is frozen, it don’t snow nearly as much and ice retreats and Earth Warms. It is that simple!
CO2 makes green things grow better while using less Water. Less CO2 would kill green things and anything that depends on green things and fresh water. It is that simple!
http://popesclimatetheory.com/
RE: Soon, the current cycles of Low Arctic Sea Ice Extents will bring down enough snow to cause Climate Theory to be reconsidered
I was thinking about just this on the drive into work this morning as NPR was reporting record snow fall in Alaska and the effort to cut through the ice to bring fuel oil to Nome.
Progressives don’t like things that they think are better Left Un-said. They hate people who try to do the Right thing. They go so far as to call their other foot Left, too. Then they just march down the same road together saying: Left, Left, Left, Left, Left, Left. They will never be drafted and it is so much easier on their mind. Forward Mosh.
Hay, this is science & the Pols, of today.
I did not say it though. Then I would be just another _______ one of them.
steven mosher, please let us see your ‘kim’ bot program. OK
Actual temperature data shows that earth is well inside the range of the last ten thousand years. There is nothing in the actual data that gives any indication that earth temperature will ever get outside of the range of the past ten thousand years.
“They also don’t like replacing p-value analysis with a Bayesian analysis because they feel this would just increase the degrees of freedom.”
The best approach should be a comparison of both. It is always nice to have somin’ somin’ to compare to somin’ somin’ :) Why that is not a standard procedure is a bit mystifying.
“But when added together these degrees of freedom allow for researchers to extract statistical significance out of almost any data set”
Is a powerful indictment.
The implications are profound.
Exactly.
It provides both the means and the opportunity. All that’s missing is a motive.
Wait, hang on..
Couldn’t agree more. Any explanation of why climate varies that requires pulling a two-digit number of non-physically-derived parameters out of a hat may well have overfitted the model to the available data.
The predictive value of a model is inversely proportional to the extent to which it has overfitted the data.
The topic has been discussed in econometrics and financial economics as “data snooping” i.e. where people build models knowing what the data looks like in advance. I have some old posts at CA on this interesting topic. Obviously paleoclimate authors know that bristlecones and Yamal have HS shapes. So their inclusion in subsequent studies “confirming” some earlier study makes the subsequent study rather unconvincing.
My own perspective on paleoclimate is, of course, heavily influenced by my skepticism of so-called systems to predict/model the stock market from other data. One can construct models that “work” for the past, but they fail “out-of-sample”. What happens (and this is sarcastically observed in the literature) is that proponents of black-box systems typically change predictors.
One of my very first observations on paleoclimate (before I got worn out in the debate) was the need to bring the proxies up to date to show that they performed out of sample. Under IPCC theory, bristlecones should be packing on ring width in the 2000s at unprecedented rates. So should other trees.
Instead of showing that these proxies actually work out-of-sample, what we’re seeing in the field for AR5 are studies that either recycle proxies with known properties (more Yamal, more bristlecones) or peculiar new “proxies” whose out-of-sample behavior hasn’t been observed e.g. varve thickness, whose properties can hardly said to be well-known. given the bizarre handling of the Tiljander sediments in literature cited on multiple occasions by IPCC AR5,
Steve McIntyre: The topic has been discussed in econometrics and financial economics as “data snooping” i.e. where people build models knowing what the data looks like in advance.
I also refer to extensive exploratory analysis as “hypothesis rescue” instead of “hypothesis test” — looking for a “statistically significant” effect that the theory might have predicted (somehow) had anyone thought of it.
There’s not a chance the tree ring proxies will be updated in our lifetimes. What if the “divergence” were shown to have continued? Who is going to risk research that might disprove their entire discipline?
Hey have you seen the latest?
The paper above, has itself, before it was even published even, been proven to be a false positive.
The authors themselves, JP Simmons, LD Nelson, U. Simonsjohn, are in fact, outright liars.
As the world renowned poet Allan Iverson so famously stated “We’re talking about psychology. I mean listen, we’re sitting here talking about psychology, not real science, not real science, not real science, but we’re talking about psychology. Not the real science that I go out there and die for and work every day like it’s my last but we’re talking about psychology man. I mean, how silly is that?”
Cranks and Cracked Pottery, indeed.
I do wonder though, how much garbage dumpster diving does one Dr. C do in any one day’s time?
Appearently enough chum can be found in those day old seafood dumpsters to keep all you denier hammerhead sharks coming back for evermore.
Autoidiocracy indeed.
“The authors themselves, JP Simmons, LD Nelson, U. Simonsjohn, are in fact, outright liars.” woohoo! Now we got a thread. What lies, what proof, who presented the proof?
One option is not to feed the idiot-troll. The other is to send uninformed Junior off to study Milikan’s oil-drop (or Feynman.s commentary on it) to see why scientists need psychology to stop themselves screwing up.
I lied. But then agian, isn’t that what this site is all about, liers telling lies
But then again, don’t take my word for it, just read the replies posted here, much funnier than South Park even.
If I said I was a climate scientist, then I’d be lying.
If I said that all those climate scientists that support the consensus on AGW, were in fact liers, then I’d be lying.
If I were to post a cartoon critical of someone else, that someone having had previously put me in my place, so to speak, and that the only motive for posting said cartoon was out of envy, or spite, or revenge. That person I would say was void of any intellect, was dishonest, could no longer peactice any real sicence in fact.
I do see though that one of the M&M’s has made an appearance though. The nutty one no less.
Anteros, Sorry I must correct you, lying, idiot, troll, not idiot troll :)
…”But then agian, isn’t that what this site is all about, liers telling lies”
I love it when a new know-it-all comes along and starts bashing the deniers. Like a bull in a china shop… no more like a drunkard on his bar stool ‘lookin fer a fight’… big talk…’yer a liar…. yer all liars…. i’ll take yall on’ wild swinging hoping something lands. He he…. after a couple weeks of ranting, they wander away, a little less sure of themselves. Just wait, after junior checks in for awhile, he too will wander off. Not to despair. A new one is right behind him. Makes me wonder if it’s just the same old troll with a new name.
Now he dunnit. Speaking for Feynman when he isn’t alive to speak for himself. Didn’t Bobby or somebody declare that a violation of some cardinal internet rule he just made up?
I do recall RF saying something like “the easiest person to fool is yourself”, though.
Good Lord!–it seems that the greenshirt flake-masters, in their desperation, have decided to prematurely unleash their latest-model, not-yet-de-bugged, weaponized “Delinquent Teenager” with this newbie, EFS-Junior monstrosity. I mean, EFS (Extra Freaking Stoopid!)-Jr.–talk about, like, a complete piece of mutant, zit-hole ejecta!
Incidentally, Anteros, some time ago there was a discussion on this blog as the best term to employ as a counter-part rejoinder to “denier.” I latched onto, but have never been fully satisfied with, “greenshirt.” Then, your genius devised that great term “doomers.” I think you’ve found just the right term. Inspired by your brilliant figure of speech, I’ve done my feeble best to come-up with some synonyms to supplement your basic term: “doom-kopf”, “doom-cluck”, “doom and doomer” (e. g. Robert and that EFS-Junior creep), “doomed-down science” (said of the the various, fear-mongering CAGW scams and hustles), and, my favorite synonym for “doomers” of them all–DOOM-BUTTS!
For what it’s worth.
Joshua, over here
What latest EFSJ? i just did a google search and nothing comes up abt S N & S. I’m with Capt Dallas – you got a link?
And Steve McIntyre – more comment!
+1. I’d like to hear more from Steve M on this topic – the many ways our ‘choices’ steer us towards results we wish for. Not just HS’s….
Could Judith suggest a guest post?
Oh my, we have a lively new troll.
And he/she appears to be in a manic phase.
You may have a future in comic relief.
“The Oceans will begin to boil…” ~Dr. James Hansen of NASA GISS
Did he say when? I could save some money on electricity and chuck my potatoes into the briny.
It sounds like it’s going to be fun (or Hansen is a complete lunatic?) :)
Anteros,
I posted a YouTube of Hansen’s speaking on dead boiling downthread.
hunter –
I saw it (perhaps at WUWT?).
It’s funny – if I’m ever tempted to get seduced by the cacophony of ‘dooming’, 5 minutes of Hansen always sets me right because he is so clearly a nutter.
Sincere, but a nutter.
I believe he was talking about if Earth experienced the same greenhouse effect the created the atmosphere on Venus, according to his theory. In order for the oceans to rise 100 meters, they couldn’t be boiling until well after the storms of his grandchildren.
Those points are not new.
I think that the only solution likely to work, if adopted, is to require that all data and all analyses be publicly posted, complete with computer code. Even that can be fudged, but since most of the documented false positives result from naivete rather than fraud, it should help the most interested of the readers evaluate the work. The publication then serves as an invitation to read the whole, or an advertisement (a word that others have suggested.)
For anyone interested in Bayesian inference methods, let me recommend a book by Francisco J. Samaniego: A comparison of Bayesian and Frequentist methods of estimation. Basically, he finds that Bayesian methods do not outperform frequentist methods unless the priors are sufficiently well-calibrated, which he says rarely happens. The book is about estimation and not testing, but the arguments carry over directly.
MattStat,
There’s one additional point, which makes me prefer the Bayesian approach: It allows for looking at the additional uncertainty that comes from choosing the prior. For estimation that’s not of much help except in specific cases, where the prior is highly skewed for strong reasons. (The strong reasons might have some frequentist basis, which would allow for reaching the same results also with frequentist approach, but the strong reasons may be also of some other nature.)
The thread about the AR4 discussion of climate sensitivity estimation is a good example of a case, where the choice of prior is crucial and the freedom in the choice affects greatly the uncertainty estimate. A change of variable would have the same effect in frequentist approach, which means that in the case of continuous variables the frequentist approach is not well defined. It falls back to something equivalent to Bayesian even, when the people doing the analysis don’t recognize that. If they don’t realize that the situation is even worse: they do choose a prior unknowingly. Thus they may also choose a very bad prior unknowingly. (It may, however, be that they do recognize through some other argument that the result is unreliable.)
My view is in any case that an honest Bayesian approach is the only honest approach, while an dishonest application of Bayesian approach is dishonest in the worst possible way. Frequentism is cheating in that view claiming some objectivity, when that’s not the case.
Nothing in what I have written is in contradiction with the observation that frequentism may very well be in practice as good or even better in estimation for a great majority of problems. People don’t usually choose really stupid variables or do other errors that would bias the results more than typical misjudgments in the choice of priors.
My main point is that no statistical error estimate is really objective, while it’s certainly possible to tell honestly, how the error estimate has been obtained giving a knowledgeable reader the possibility of judging the new evidence without extra bias.
This explains the power of the Maximum Entropy approach, which straddles the characteristics of the Frequentist and Bayesian approaches. For MaxEnt, you need known constraints and the rest gets filled in automatically. The constraints of the system are often known based on frequentist empirical data (such as a mean value), while the higher-order moments are given the least bias and maximum uncertainty due to the natural entropy mechanism.
Thus, the natural subjectivity of the Bayes approach is reduced and that criticism is deflected. Only the information that you have available and the constraints that you know about get incorporated, giving the most objective view possible.
MaxEnt can thus handle Normal statistics, Markov processes, and fat-tail statistics without batting an eye, while frequentist thinking is really locked into a Normal world view.
This of course comes from my own practical application of probability and statistics for environmental modeling, and the academic statisticians can differ on the formal rigor involved.
BTW, my last statement is aimed at MattStat because he does have the breadth of knowledge in the field that I can only slowly catch up to.
WHT,
On one of the main points of my comment MaxEnt is not any better: The value of entropy is not objectively calculable from data. Calculating the entropy requires the definition of the measure used for comparing phase space volumes.
That’s a problem that gets small together with all other problems. In other words defineing the measure is a serious problem as long as any reasonable approach has serious problems. There are no ways around this fundamental problem – unless we accept a theory that defines the measure. That works well in Quantum Mechanics, where the basic theory really gives the answer, but the problem is unavoidable in statistical data analysis in absence of a strong theory. (For a genuinely discrete system the problem may be absent, but discretizing doesn’t help, when real variables are continuous or the total number of possible states is infinite.
As someone who sees Bayesian approaches as valuable, my perspective has been colored by the fact that my largest experience with Bayesian approaches involves a field, biomedicine, where priors can often be quantified objectively on the basis of recorded data. If, for example, we want to know the probability that a man with a PSA value exceeding 4 ng/ml has biologically significant prostate cancer, we can estimate this, utilizing available data on the prevalence of such values in prostate cancer, their prevalence in the general population, and the prevalence of prostate cancer. In fact, without those priors, the test would be uninterpretable. The available data are subject to error, of course, but they are far better than nothing.
The troubling issue for Bayesian inference is subjectivity, but I see that as basically another way of saying that the use of expert subjective prior estimates implicitly involves the expert’s sense of what the objective value of the priors is likely to be, based on his or her experience. The question then becomes – is this better than nothing? Or worse? As Pekka has pointed out, frequentist approaches implicitly assume values for priors without stating that; therefore, while subjectivity may limit the confidence we can apply to Bayesian-based estimates, I see no reason why we shouldn’t have even less confidence in estimates where the subjectivity is hidden rather than acknowledged. This is probably true in particular for scientists and others who don’t pay much attention to these issues, and who often interpret a p value less than 0.05, for example, as showing that there is less than a 5% chance an observed result was due to chance.
Ideally, I believe, it would be best for those applying subjective priors to describe how they arrived at their estimate. This would help them to be more careful in what they estimate, and help the rest of us decide how much confidence to put into the posterior probability values that emerge. It would also help us (and them) discern when estimates are based on circular reasoning, whereby a prior PDF range is assumed, and the posterior values that emerge are then subsequently used as priors in later estimates.
Pekka says that MaxEnt needs models.
I agree and almost always work in the context of conventional physics models. MaxEnt can then estimate the aleatory uncertainty in the various model parameters.
This distinguishes MaxEnt from MEP, the latter of which is an informal stand-in for a real physical process. I have less confidence in applying a cookie-cutter approach to how entropy is produced during a dynamic process.
In the above message I wrote:
(The strong reasons might have some frequentist basis, which would allow for reaching the same results also with frequentist approach, but the strong reasons may be also of some other nature.)
By that I referred to the same observation that MattStat made on Fred’s comment, but I referred also to the other part of the difference: the other strong reasons. The choice of the state space measure is an essential part of the subjective contribution in every analysis that expresses its results as probabilities or likelihoods for the parameter being studied, only carefully described conditional probabilities are fundamentally more objective (and even they hardly perfect in practice).
Both approaches may provide the same information, when properly presented. Thus the issue may often be more a matter of style than matter of substance in practice. One example related to presenting the data is that of Fred. In my view the Bayesian formulation is more natural for that when contrasted to, how the same information is presented by frequentists.
FredMoolton: If, for example, we want to know the probability that a man with a PSA value exceeding 4 ng/ml has biologically significant prostate cancer, we can estimate this, utilizing available data on the prevalence of such values in prostate cancer, their prevalence in the general population, and the prevalence of prostate cancer. In fact, without those priors, the test would be uninterpretable.
Bayesian inference in medical diagnosis is frequentist in intent. Empirical Bayes estimation and testing in hierarchical models is also frequentist (I don’t know who said it, but “There is nothing Bayesian about empirical Bayes”.) That is, the prior probability distribution is a model of the frequency of occurence of measurable attributes in an objective population of entities — people, in your example — and the conditional distributions refer to the distributions within subsets of the population. The assumption of the prior distribution is potentially testable by computing the cdf from a large enough sample.
Fred, Pekka and WHT:
To me the main weaknesses of Bayesian inference are: (1) that it downweights the sharable information about the shared world (the measurements and their sampling distribution and their likelihood), by directing attention to the non-shareable, non-testable prior and (2) there is no sense in which the prior distribution can represent the marginal distribution of the unknown, so the claim that the posterior distribution can be the conditional distribution of the unknown given the data is unsupportable. This is different from the Empirical Bayes modeling of hierarchical designs.
WHT: BTW, my last statement is aimed at MattStat because he does have the breadth of knowledge in the field that I can only slowly catch up to. Thank you for the compliment. I am doubtful that I deserve it, but it was nice to read.
That matches what I have thought, as the frequentist and Bayesian results one obtains usually line up if the same empirical priors get used.
Capt Dallas: Re: Shorts – you can express an opinion that avoids the question – ‘I don’t think it has anything to do with the shorts”.
That didn’t work :) “I like my women chunky” was pretty transparent. “Lift your back fat so I can see it”, created an ominous, but rather pleasing silence for the second half of a game. “You betcha!” works like a charm. It gives the appearance of humor while not placing too much blame on the shorts :)
You should try a “skeptic” standard technique when they’ve been asked a question they don’t want to answer.
Look!!! Squirrel!!!!!
I should add that, in the grant-funded research that I participated in, the grant proposals (and IRB submissions) specified the analysis plan in much detail, including the sequence of hypothesis tests and model-building (authors’ recommendation #6) and all of the covariates were justified by the results of prior research. Where we did “exploratory data analysis”, we did the hypothesis tests first, and clearly labeled all the post-hoc tests as such. All of the tests were referred to (so that readers could keep track of the total) in the papers submitted for publication. There is a movement in the US toward requiring that all clinical research have the protocols publicly archived before the studies are begun. As I understand it ( I may have to check up on this) the FDA will not accept a study report in support of a drug unless the protocol was publicly archived before the study began; protocols not followed by research reports are judged to have produced non-supportive results.
Emphasis on parameter estimation and effect-size estimation will not solve the basic problem: somewhere along the line someone has to decide what to do next: that decision will be based on the ratio of a parameter estimate to its s.e., or some such, and hence the decision will be subject to random variation and all the “experimenter degrees of freedom” listed.
“The Himalayan glaciers are retreating so fast that the rivers may dry up in the summer by 2040… catastrophic.” ~George Monbiot
Has anyone seen the cover of the book, “Heat: How to Stop the Planet From Burning,” by George Monbiot
The cover shows a river that is running red like lava.
What about a final eliminator between Monbiot and Hansen’s ‘Storms of my grandchildren’ with the winner to take on the one-and-only M.Mann’s ‘Dire Predictions’ [No hint of irony] aka ‘It’s gonna be so much worse than we thought’ aka ‘we’re all gonna die!!’.
:)
I’m betting on alien invasion before any of this stuff has a chance to happen.
What I’m uncertain of is whether it will be nasty aliens out for conquest or the benevolent kind that rid earth of those destructive humans.
(Ok, I did watch The Day the Earth Stood Still last night.)
“We know that climate change is happening.
“We know it could, if the worst predictions come true, destroy the conditions which make human life possible.
“Only one question is now worth asking:
“Can it be stopped?”
~George Monbiot
Dr. Curry et alia,
In Economics, the well-known classic in this genre is Ed Leamer’s “Let’s Take the Con out of Econometrics,” which is still a delightful read after all these years:
http://www.international.ucla.edu/media/files/Leamer_article.pdf
There is also Ioannides’ great article about the “institutional” perspective on why “Most Published Research Findings are False,” which I always assign in my advanced stats and design class:
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
What’s that great statistician joke?
“Logic is a systematic method for getting the wrong conclusion with confidence. Statistics is a systematic method for getting the wrong conclusion with 95% confidence.”
http://www.workjoke.com/statisticians-jokes.html
1. Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.
This rule might prevent an author from sequentially collecting data until the null is rejected, but it is an unusual situation. And authors can claim, after the fact, that their subjective stopping rule is what they had in mind all along. Who would know?
2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data- collection justification.
Seems like an arbitrary number—why 20? This reflects an erroneous view that the likelihood of false positives depends on the sample size (which is true of false negatives).
3. Authors must list all variables collected in a study.
This can run into the hundreds or more. Better to make this a part of an electronic resource that the author is required to make available for download (including the data).
4. Authors must report all experimental conditions, including failed manipulations.
This is sensible.
5. If observations are eliminated, authors must also report what the statistical results are if those observations are included.
Agree, unless the observations are shown to be erroneous or improperly collected.
6. If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate.
I can’t think of any principle of good statistical practice that validates this idea.
The main lesson from this is to avoid statistical fundamentalism – that the stats results on their own are ‘truth’.
Very useful tools, but don’t get carried away in over-interpretation. It’s no surprise that the original article comes from the psych feild – it’s very easy to find what you want to find.
Physical sciences are somewhat more constrained by the need for results to conform to observed reality.
Denizens should take note; analyses that purport to show no warming or explain it via natural cycles based on 3rd order (or higher) polynomials are hereby called on it.
Sadly, I suspect that some denizens will quickly loose their current enthusiasm for this lesson.
Michael –
It’s not the denizens you need to worry about. You should pop over and disabuse Kevin ‘where’s all the heat gone’ Trenberth of the notion that warming has come to a grinding halt.
Strangely, Phil ‘I’ve got 25 minutes invested in this excel spreadsheet’ Jones has also been infected with the ‘it’s stopped warming’ delusion.
Help them out would you?
Sure, as long as you help out those convinced by the ‘no warming’ based on high order polynomial mis-fitting.
Funny thing is – I would do. Can you point me towards some of them? As it happens I would send them over to Roy Spencer’s blog to remind them that higher order polynomials are there for a laugh.
As for Trenberth, Jones et al [AR5 – “not statistically different from zero”] you have your work cut out. They really seem to have got it into their heads that warming has gone AWOL. Good luck :)
It may be helpful to revisit a scientist whose hypothesis was as controversial and difficult to demonstrate then as the disputed hypotheses of attribution and climate sensitivity are today. I reread Darwin’s 1859 Origen of Species again this autumn. I urge everyone to read or reread it for an example of how a brilliant scientist can be modest, circumspect, and forthright in fleshing out all the uncertainties and problems with his hypothesis so that others may better evaluate and test it.
There are major differences between what Darwin did and climate scientists are doing, I know. For one thing, there was no urgency of possible catastrophy (except for his reputation). Also Darwin suffered from “non-conformation” bias. He knew that his hypothesis would make him unpopular (or worse!), even in his own community and family. There are similarities, too. Mendel’s work was unknown, so there was no mechanism to explain the workings of natural selection, and Darwin acknowledged this. I think this is analogious to how little is understood about the all-important feedback mechanisms in climate science. Another analogy is the time factor. Just as it is impossible to test climate model projections over short time periods, so it was impossible (until much, much later when fruit fly genetic changes were studied) to test hypotheses of selection and variability.
Besides humility, Darwin did something I see too little of in climate science. Darwin amassed and catalogued vast amounts of data in ways that others could inspect and critique his work. He shows multiple lines of evidence to support his thesis from the work of pigeon fanciers and animal husbandry to the tens of thousands of observations he made in his field work. He spent years considering the meanings of his observations. I consider him a giant in the field of science for many reasons, but not the least, because of his care and modesty in considering the evidence.
I’ll concur with that. I’ll especially agree that there is a huge difference in (alleged) urgency which explains a fair amount of the shoddy science.
Remarkable too that he had the theory in his head for 20 years while he patiently collected data.
Don’t you think Michael Mann compares well in the way of modesty and humility?
LOL – lovely example of confirmation bias there.
Pity that reality diagrees with you.
Darwin wasn’t sure of his theory, but then heard that Alfred Russell Wallace was working on something similiar (in fact he had already published a short paper that could properly be considered the origin of the natural selection theory) , so he hurriedly threw the book together and published before Wallace did.
Michael –
Reality very much agrees with me – and strange that you would suggest otherwise when the facts are available for all to see. Darwin had the theory in the thirties – long before Wallace – something Wallace acknowledged as soon as he met Darwin. When Wallace’s ideas were presented to the Linnaen society in 1858, they were accompanied by the essay written by Darwin for Hooker eleven years before.
There was never any contemporary doubt – from Wallace or anyone else – that Darwin’s was the earlier [and much more comprehensive] version of the theory.
Darwin’s not being sure of his theory is a credit only to his scientific scepticism and reticence. In today’s world, with today’s mentality he would have published his theory 20 years previously when he first thought of it (and when it would have been completely new to Wallace)
I’m slightly baffled as to why you mention confirmation bias. I’d been studying Darwin’s life and ideas for ten years before I became interested in climatology so I think you perhaps have it the wrong way round.
When you say that Wallace’s short paper could properly be considered the origin of the natural selection theory I think the answer is, ‘bullshit’
Michael –
Very strange that you should suggest that reality doesn’t agree with me, when patently it does – and the evidence is there for everybody to see.
Embarrassingly, you haven’t the faintest idea what you’re talking about. When Wallace had his ideas presented to the Linnaen society in 1858 [at the behest of Darwin himself] they were accompanied by an essay Darwin gave to Hooker eleven years previously. There was never any contemporary doubt – from Wallace or anybody else – that Darwins was the much earlier (and more comprehensive) theory. Wallace admitted exactly that when he met Darwin.
With today’s mentality Darwin would have published his ideas 20 years earlier – when they would have been completely novel to Wallace. It is credit to his scientific scepticism that he didn’t.
I don’t know where you get the fantasy of ‘confirmation bias’ from – projection, perhaps. My studies of Darwin and evolutionary ideas preceded my interest in climate by about a decade, so I think you have it the wrong way round.
Oh Ant, how charmingly naive of you!
Wallace was about to give the first public scientific presentation of the natural selection theory, that Darwin had thought of as his own.
The best Darwin could do was to present his private letter – it was a piece of territory-marking that Wallace was gentlemanly enough to countenence. Someone more interested in their own career would have told him to go jump.
Michael –
History renders your attempts at distortion quite comical.
You say –
“Wallace was about to give the first public scientific presentation of the natural selection theory
Really? He was thousands of miles away in Malaysia and didn’t return for over a year. He was in no position to present anything to anybody and neither did he even suggest publishing his ideas.
“The best Darwin could do was to present his private letter – it was a piece of territory-marking that Wallace was gentlemanly enough to countenence”
Utter garbage. Wallace, being on the other side of the world didn’t countenance anything. Actually, neither did Darwin – it’s ludicrous to suggest he ‘presented his private letter’ as he wasn’t even there! His 220 page essay from more than a decade earlier was excerpted by Lyell and Hooker – of their own volition – to show that Wallace had belatedly stumbled on something Darwin had thought of years previously.
The biggest boost to Wallace’s career was to hang on to Darwin’s coat-tails. He was forever – justifiably – grateful.
Any other history you’d like to catch up on?
No, there is still dangerous warming. We are just in the “hiatus” stage. We will know the “hiatus” is over, when they have a successful CO2 mitigation junket. I predict that will be Detroit, 2032.
Ant,.
You’ve got your history on this totally wrong.
Wallace sent his latest manuscript to Lyell and Darwin for their comments before he published it.
They recognised the signifcance of it immediately, and so arranged for Darwins earlier private treatise to be presented at the same time as Wallace’s.
Who knows when Darwin would have published if not for Wallace’s paper.
Might also be worth reminding people that the first discussions of climate and mans possible impact on it, predates Darwins’ book by more than 30 years.
Investigation of climate sensitivity goes back more than a hundred years.
And the vast collection of climate data we have makes Darwin’s effort look miniscule.
Darwin’s effort was actually science. Anybody can gather a vast collection of inadequate data, adjust it, smooth it, homogenize it, apply systematic confirmation bias and come up with an unconvincing theory. What evidence do I have that the CAGW dogma is unconvincing: Copenhagen, Cancun, Durban. When they get to Detroit, you will know for sure that it’s all over.
Discussions of man impact on climate goes back even further, I imagine to those who perform rain dances.
Your observation is utterly beside the point.
Question: BCPs are the principle ingredient in getting a mannian shaped hockey stick. Theory says, since 1960 they should have put on rings like gangbusters. It is a simple mater to shut certain skeptics up about this issue. Go update the proxies. 50 more years of data..
There is no credible explanation why this basic task has not be completed.
care to offer one? or will you change the topic.
Michael –
Your history is painfully shaky. Discussions of man’s possible impact on climate have been going on for millennia, and to think that the first discussions of climate predated Darwin’s book by 30 years forgets that climate has been a central topic of discussion since the dawn of civilisation.
You might ponder a previous age of superstition and alarmism – the depths of the Little Ice Age in Europe, where 50,000 people were executed – many for their alleged disruption of the climate.
Climate is not a new topic – and neither is alarmism.
Doug was angling on the old and tested Darwin’s theory vs. the all-to-new AGW theory.
Just another error cascade that needs to be knocked on the head – pity all the skeptics could contribute was stupidity about rain dances.
Michael.
Do some reading. I am not a skeptic. I believe in AGW. GHGs cause warming. I believe the IPCC has a good estimate on sensitivity, but they are a bit on the high side.
So, not a skeptic. I do not however accept every line of bullshit that my tribesmen shout. Like your nonsense
Steven Mosher –
It’s instructive for us all that people (Joshua particularly) take you so easily for a “skeptic”, just because some of your ire is directed at the bullshit from your own side.
You do bring up the issue of labels for me, though. cf – I believe in AGW.[tick] GHGs cause warming.[tick] The IPCC has a vague idea of sensitivity but their estimate will fall a little over time……[tick]
…..And I’m a sceptic.
I’m sceptical that warmer is likely to be noticeably negative. I’m sceptical that ‘change’ is a negative thing. I’m sceptical that life and humanity won’t be able to happily adapt. I’m sceptical that deliberately not using fossil fuels is anything but an idiot idea that’s never going to happen – and would be a bad idea even if it did happen. I’m sceptical that anyone is vulnerable to climate change – though they may be vulnerable to climate. I’m sceptical that many people have any reason or evidence to be concerned about future levels of ‘windiness’ or ‘raininess’ – apart from a fear of the dark, the unknown or change, and those fears are a common feature of humanity when there’s nothing else to be concerned about (ie war/disease/hunger).
I’m a realist about the slight warming of the 80’s and 90’s but very much sceptical of doom, apocalypse and the end of life on earth.
And FWIW, a large portion of ‘sceptic’ argument is politically motivated drivel.
I just don’t have the energy or enthusiasm (or balls) to attack my tribesmen.
Steven Mosher –
The fact that you a frequently taken for a sceptic (particularly by Joshua) should be instructive for us all. Obviously not enough of us call out the crap from our own side – so it confuses when it occurs.
You also bring up the issue of labels. cf. I believe in AGW [tick] GHGs cause warming [tick] The IPCC has a vague idea of sensitivity although their estimate may fall over time [tick]
And I’m a sceptic.
I’m sceptical that warming will be negative. I’m sceptical that change is bad. I’m sceptical that life and civilisation won’t be able to (happily) adapt. I’m sceptical about all visions of doom and apocalypse. I’m sceptical that deliberately not using fossil fuels isn’t a stupid idea – and something that isn’t going to happen. I’m sceptical that not using fossil fuels won’t make things very much worse. Well – and all the other things that slip my mind for the moment.
I find that the majority of sceptics that aren’t politically motivated think pretty much like I do….. and we tick the three questions you posed [with some variation on the last].
Thing is, I don’t have the energy or enthusiasm (or balls) to call out the reams of sceptical crap from my own side. The nonsense and junk science and paranoid politicking. Which come to think of it is maybe reasonable – the disparate sceptical mass doesn’t need to have the rubbish sheared off, whereas maybe a consensus for action does?
steven,
the point was Don’s nonsense about Darwin. We have far more data on climate than poor old Darwin could have collected in a score of lifetimes.
BCPs: stop obsessing. You don’t need them for hockey stick.
“…forgets that climate has been a central topic of discussion since the dawn of civilisation” – Anteros
Oh lord, what drive
l.
Hmmm, nice typing.
Doug Allen’s attack is flimsy gauze. Pretty good except he should have added another dead scientist or two who cannot wipe off all the tongue juice and defend himself from his enlistment into a cause perhaps not of his choosing.
Feynman, Darwin, Galileo, etc.. You fill the stand with witnesses with no heartbeat. To get a beating heart, it’s Fred Singer time.
So now someone thinks that by rewriting the history of evolutionary science to put Wallace, not Darwin, at the head proves today’s skeptics of AGW are wrong?
Please clarify this one for me. It is more sublime than even a tag time troll attack by the two trolls themselves.
http://judithcurry.com/2012/01/12/false-positives/#comment-158400
mike,
All too glad to get under your skin. But seeing as you’re so obviously thin skinned … note to self … chum of any type is a whole lot of fun.
Like Maxwell in the GEICO commercials …
Wheeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee …
Something bothering you?
ROTFLMFAO!
O. K.. I’ve figured it out. I’m dealing with a goofy kid. Past your bed-time, Junior!
I thought Judith had a rule banning those (with a mental age of) under ten?
It takes a disciplined constitution not to start to trash talk when faced with the ignorance and crackpot theories of commenters on this blog.
I always figure that real world people have an easier release mechanism than do commenters — consider that engineering professors can FAIL half the class to weed out the kids that clearly can’t cut it. But within the magical universe of a blog, you just have to sit back and endure a 10-year-old such as Joe LaLonde describe his crayon creations.
I have adapted to roll my eyes on most of the comments, and my release mechanism is writing something like this every once in a while to remind people of business reality. We can’t fail or fire anyone in this kind of environment, otherwise the proprietor gets accused of limiting free speech. Because, as everyone knows, the truth has to be fair and balanced.
BTW, EFS_Junior knows his stuff, and you all can’t take trash talk.
WebHub,
You say, “BTW, EFS_Junior knows his stuff…”
So, then, I guess you must be Junior’s mom, right WebHub? Well, hate to say it, but that li’ll darlin’ of yours is sure a bratty pest, Mommy dearest. Though I suspect you think him “cute.” You’re spoilin’ him, Ma.
Nice trash talking there, Mike.
Water off a duck’s back.
WEB,
Just curious – what stuff is that?
http://judithcurry.com/2012/01/12/false-positives/#comment-158388
John Carpenter see;
http://judithcurry.com/2012/01/12/false-positives/#comment-158428
Wheeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee …
I tend to concur with the comment by cwon14 (January 12, 2012 at 3:13 pm): in the case of AGW we do not deal only with involuntary bias, but with “militant research”, i.e. research with an agenda. The agenda does not lead to involuntary but to voluntary bias: some have even stated that predicting a good disaster could be very convenient for the cause, and have also tried not to “dilute the message” by reporting annoying uncertainties or acknowledging contrary views.
Both militancy (or more pruriently, “advocacy”) and involuntary bias are problems in science. When a catholic doctor investigates (and finds) condom leakages that would invalidate the use of condom as prevention of sexually transmitted diseases, the good doctor is not only being led by his faith into involuntary bias and wishful thinking: choice of subject as well as choice of procedures and rules for acceptance of conclusions would be influenced (should we say tainted) by his/her preconceived conclusion (the case is an authentic one, which gave the author some 15 minutes of fame a few years ago). A card-carrying member of Greenpeace finding through complex scientific investigations that all manner of horrific climatic disasters would befall mankind lest it amends its ways is not something hard to expect. How much is voluntary and how much isn’t would depend on the individual case considered.
Hector M –
Interesting. Isn’t it the case that the voluntary and the involuntary overlap and shift in their influence? I think that’s why we’re all rightly sceptical of the funding that some people have for their research – on either side of the debate.
Even those whose funding is above scrutiny can give signs that their agendas will be influencing all the thinking and ‘seeing’ of ‘facts’ that they will be doing. When Phil Jones hopes for an El Nino to ‘wipe the smile off the faces of the sceptics’ it isn’t hard to imagine that having a little influence on his judgement. In fact, from that point on, is it conceivable for him to carry out any science at all?
I tend to agree with both the funding and involuntary bias comments. Recently, people have used the BEST data to show that temperatures have risen recently, or not risen, based on the timeframe chosen. Ironically, the same criteria is being used today to dismiss the flat trend that was used two decades ago to dismiss the warming trend, namely that the timeframe is too short to be significant at the 95% level.
Those in the militant warming camp (to use Hector’s reference) will point to 2010 as the warmest year on record, while those in the other militant camp point to 2008 and 2011 as the two coldest years this millenium. Not to mention, the choice of datasets which best supports their viewpoint. There are sufficient ‘facts’ to support either viewpoint, but neither is telling the whole story. To return to earlier posts, these could be called ‘lies of omission.’
Interesting Dan H.. Is leaving out that part of the ocean that lies below 700 meters an omission?
If natural ocean cycles indicate global cooling, Keenlyside et al, and warming holds it off, what sort of thing is it to claim no warming?
JCH, talking of what lies, why not ask the your Joshua bot, Don.
BTW, WTF are “bots,” anyway?
go to cleverbot.com and ask
‘what is a bot’
Thanks for asking Josh.
Guess I’m going to have to go over to the site Steve recommends, as I don’t have a clue what a bot is.
But since people claim kim is a bot, I’m sort of hoping a bot is another term for hot dark-eyed chick.
I just thought of this-
you might say Darwin was a model scientist compared to our current modelling scientists!
“Global warming has melted the polar ice caps, raised the levels of the oceans and flooded the earth’s great cities. Despite its evident prosperity, New Jersey is scarcely Utopia.” (Godfried Danneels)
Weather or not…here we come:
“All across the world, in every kind of environment and region known to man, increasingly dangerous weather patterns and devastating storms are abruptly putting an end to the long-running debate over whether or not climate change is real. Not only is it real, it’s here, and its effects are giving rise to a frighteningly new global phenomenon: the man-made natural disaster.” (Barack Obama)
Wasn’t the the guy who claimed that under his enlightened rule the Earth would cool, the oceans recede, and coal companies would go bankrupt?
The calm before the calm…
“Some of the scientists, I believe, haven’t they been changing their opinion a little bit on global warming? There’s a lot of differing opinions and before we react I think it’s best to have the full accounting, full understanding of what’s taking place.” (George W Bush)
8 hours ago … Calif. economy slips to 9th largest economy behind Brazil. Next, it will fall to India.
You guys should look on the bright side. From having the biggest empire the world has ever seen (and being your colonial rulers), we’ve just had our whole economy overtaken by Brazil. You’re worried about that happening to one of your 50 states? There’s nearly 200 million of those soccer-playing beach-lovers!! Take it easy!! :)
Leftist Economics 101: Causes the glass to be half empty then mandate a smaller glass.
Wisest, most sober academic statistician, whose core specialization was fundamental concepts: “Too much of this p-value stuff.”
Judith,
The system we currently have was created to be bias by the enclosed system to protect itself. Protect the tradition of passed down laws of science to protect the founders which insist on following certain routines. This is instilled in the funding of science that you have to be an economist which then distracts from the researching end in the enclosed areas of study.
Lord help if the research happens to lead to another area that is being followed and not being paid for.
Time lines are sciences big Achilles heel as where you pick it becomes a different conclusion. Who starts at creation?
Would that not be a smarter move to understanding this planet?
Much of current conclusions are just guesses shaped by the society of the day.
Who actually is foolish enough to start all over with evidence instead of all these theories?
I am the foolish one.
Scientists quote garbage theories that I challenge or they just completely ignore what is actual physical evidence in hopes that it will go away and their enclosed system is safe.
Who asks how is the temperature created for the data?
Is it uniformed in it’s collection for every individual area?
Why is the temperature anomalies applied to the whole planet when only certain areas at that time generated it?
Who is responsible for science to be pure?
Certainly not our current consensus of scientists, they a biased for their funding and job positions.
Judith Curry
The “false positive” paper by Simmons et al. is a psychological extension of the “paradigm” treatise by Thomas Kuhn.
How does this apply to climate science today?
The authors start off with the statement:
But has this really been the ”job” of the IPCC consensus process and the “mainstream insiders” who run this process or has it rather been to find the proof for the scientific support for a political agenda?
Let’s leave aside cases of “agenda driven science”, i.e. fraudulent massaging of data or willful exclusion of reports or data, which do not support the preconceived hypothesis or the agenda. IPCC has been guilty of this in several instances in its AR4 report, which have been documented.
http://sites.google.com/site/globalwarmingquestions/ipcc
But, even assuming that the climate scientists involved are honestly attempting to discover the truth about what makes our planet’s climate behave as it does – they will discard data points that lie outside the paradigm as “outliers” or “noise”, or, in extreme cases, according to Kuhn, be physically unable to see these data points, which lie “outside the box”.
This has been described as “paradigm paralysis”.
In Kuhn’s treatise this is only “cured” by a “paradigm shift” – whereby the old paradigm is painfully replaced with a new one. This shift can happen slowly, but usually there is some sort of a scientific breakthrough, which triggers it. And the new paradigm often comes from totally outside the specific scientific discipline, which created the old paradigm in the first place.
But back to the Simmons et al. paper.
The authors tell us that part of the problem is that researchers have to decide which avenues they want to explore and report:
The authors add:
They discuss how scientists treat “outliers”, giving examples that follow the behavior as described by Kuhn.
Researcher ”degrees of freedom” are discussed, as they ” influence the probability of a false-positive result”.
Obviously, the ”IPCC consensus process” constrains the ”degrees of freedom” for climate researchers today.
The authors show that the likelihood of a ”false positive” decreases with a larger number of data points.
In order to address this problem, Simmons et al. propose “six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process”.
For authors:
The authors are describing the reporting of actual experimental results or physical observations. Most climate reports, however, are not based on reproducible experimentation or physical observations, but rather on model simulations.
I would add that another prerequisite be that in such cases the authors must clearly state that ”the results reported and the conclusions reached are not based on actual physical observations or reproducible experimentation, but instead on model simulations”.
For reviewers:
As was shown in the case of peer review of climate studies, this often became ”pal review”, where a group of like-minded colleagues simply rubberstamped a questionable paper, which showed the desired result.
In addition, IPCC “reviewers” were often political representatives, who simply wanted to ensure that their political agenda was supported by the scientific study.
So I would add another requirement, namely that reviewers be truly independent, coming from outside the inner circle of scientists, preferably even scientists who are skeptical of the “consensus” premise of CAGW.
The authors add:
In climate science today a ”successful” study is one that confirms the ”consensus view (of potentially catastrophic AGW)”.
In addition, as pointed out above, the ”experiments” are usually NOT physical experiments, in the true scientific sense, but rather ”model simulations”.
So the ”experiments that work” are those model runs, which confirm the ”consensus view (of potentially catastrophic AGW)”, while those ”that do not work” (and are not reported as a result) are those, which do not support this premise.
Getting the political bias out of climate science today will not be achieved simply by tightening up on “false positive” reports, as proposed by the authors.
But it could be one step in that direction.
Max
Briggs has been pounding this drum for years.
People see what they want to see, especially when they have a bee in their bonnet. Check out the brilliant bee graph in:
http://www-personal.umich.edu/~jpboyd/sciviz_1_graphbadly.pdf
As Anteros pointed out above, Chris Colose came into science with a bias, and many younger ‘climate scientists’ are a self-selecting group (or selected by the Universities who hope to get funding) specifically because they are AGW believers. What Univeristy would appoint a sceptic? Governments would cease funding if they are told ‘Climate is not a problem. Nothing to see here. Move along.’
Sadly, soon enough, the statement “98% of climate scientists believe in AGW” will come true, because that’s why they got into the business in the first place. Scientists creating false positives will be endemic, while Nature increasingly diverges from their ‘ex cathedra’ statements, and the public will be left wondering what is going on. This will damage the credibility of science as a whole.
Perhaps, Dr Curry, we could persuade you not to retire anytime soon? :-)
First of all, you have perfectly described Girma’s graphing techniques.
Everyone with any worth comes into science with a bias — that bias is that they believe that they will succeed, first in academia and then in a career capacity. The ones that don’t have that drive and motivation are unnecessary, or at best, replaceable cogs.
That may reveal a dilemma of perceived bias, but you have to take the good with the bad. The people with the most brilliant ideas may also happen to have the most egotistical view of their own capacity for achievement. This does not damage the credibility of science because this is the way it has always been.
Web,
Chris’ bias is a whole different sort of bias.
Credibility of science….hmmmmmm.
What are actual facts then?
Hmm. Where has Girma gone recently? :-)
Every individual may have a bias. My point stands: if the bias is one-way and self-selecting in a subject matter it will become self-reinforcing. It may end up as a pseudo-religion or something approaching mob-psychology.
We’re more than half-way there, as most sceptics are bloggers, and not because they are incapable of writing a scientific paper.
If things were more balanced in the science, I’d suggest that one solution to false positives is to have every pro-AGW paper reviewed by 2 sceptics and 1 believer; and vice-versa.
Sadly, the imbalance makes this impractical, and it can only get worse.
cui bono,
The ying and yang of keeping balance or finding an exactly repeating pattern in a system that is in constant change.
Yet scientists still are looking for that balance that does not exist.
Joe, I’m talking about the balance of opinions (or bias) in the Climate Science community. Energy balance questions are for a more technical thread and more technical folks than me.
He is back and he posted a graph in the prior thread:
http://judithcurry.com/2012/01/11/geoengineering-for-decision-makers/#comment-158180
This shows all the problems with people manipulating graphs that your link describes — Girma intentionally compressing the graph, thus visually flattening the line, etc.
You were the one that brought this issue up and this problem of cherry-picked graphs is most acute among the skeptic community.
Web, I certainly don’t agree this is mainly a ‘skeptic’ issue.
David Whitehouse has just won a bet made with AGW believer James Annan on whether Hadcrut would show a warmer year than 1998 by this year.
He says: “Writing shortly after the wager was placed James Annan said he believed it was a fairly safe bet, though not certain, as the trend since the current warming spell began, around 1980, was upward (showing those same trendlines!) He drew a straight line from 1980 to 2007 and projected it forwards concluding that sometime over the next few years HadCrut3 would rise above its highest point which was in 1998 (a strong El Nino year.)”
Sounds like Annan had a touch of the Girmas!
http://www.thegwpf.org/the-observatory/4748-winning-a-climate-bet.html
Well, put your money where your mouth is and tell Girma off. He is on this blog, after all, unlike Annan, and we can then all ostracize him for making crap up.
Yet, I don’t think you will do this, because Girma supports your overall view. Hypocrites.
cui bono
Went back home to Ethiopia for a family visit during the holiday.
Web –
I’ve never quite understood why Girma’s little graph gets you so agitated. Why take it so seriously? It is only the equivalent of the nutters on the other side proclaiming that the world will end at 2.37 tomorrow afternoon.
Chris Colose’s bias is more serious because he is a professional scientist – except that precious little science can occur when an all-pervading conviction influences every choice, every interpretation, every ‘reading’ of the evidence.
You may not like ‘anecdotes’ but how do you feel about the psychological implications of Milikan’s oil drop and the subsequent history? Do you not see the possibility of the same thing occurring in a very widespread way in climate science? With all the extra pressures of funding, groupthink, wagon-circling, band-wagon activism and fundamentalist hysteria [eg Hansen] etc?
I know you like the discrete and dichotomous world of hard science, but didn’t the overall ‘flavour’ of the climategate emails raise the kind of concerns that are addressed in this paper? – “these degrees of freedom allow for researchers to extract statistical significance out of almost any data set” This is particularly true in the messy, assumption-laden world of climate ‘science’.
If those repeating Milikan’s experiment were so distorted by the minor pressures of the circumstances they were in, how much more distortion can be attributed to an area where whole world views are at stake? I’d go so far as to say it was inevitable that someone would create a Hockey Stick graph – irrespective of what the whole mass of relevant data would say if it were asked merely to speak its truth.
Unwitting bias is important. Girma’s funny graph isn’t!
Anteros – My understanding of the Milikan oil drop results was that his value for electron charge was off (slightly), and the error remained uncorrected for some time because there was an unwillingness to challenge the earlier results. I see exactly the opposite happening today – almost everything important that emanates from mainstream climate science is immediately subjected to challenge. This certainly doesn’t imply that false positives are less likely than previously,, but I think it does suggest that results that are truly false don’t remain widely accepted for long. Of course, whether a positive is a true or false positive is something determinable only in retrospect, and therefore many “positive” results today based on one or a few studies remain in limbo. I don’t think that’s true of conclusions verified from many different angles, but we may not know for a few years.
The example of Mann’s hockey stick has been raised. We have to be careful here what we refer to as the “positive” conclusion. His general conclusions about temperature trends for more than the past thousand years remain a matter of disagreement, with some studies supportive and others contradicting those conclusions. On the other hand, if the “positive” result is that his original method sufficed to prove his conclusions with high confidence, I would suggest that it didn’t take an inordinately long time for that to be regarded as probably false by many scientists, perhaps a majority. Whether Mann himself is convinced is not particularly relevant.
Anteros – I believe Chris Colose is still in college.
JCH –
Thanks for pointing that out. I just made a wild guess that he’d have finished his PhD by now. I stand corrected.
No need to guess
http://blog.timesunion.com/weather/author/chriscolose/page/2/
A false positive anecdote, congrats Ant.
So Colose shows some initiative while still in school — starting a blog, collaborating on critical reviews, writing weather columns. Good for him and his future prospects.
Fred Moolten –
I take your point about the Hockey Stick – it may even be that the ‘backlash’ had an unjustifiably large effect, though I doubt it. I agree too, that there are challenges to most findings (unlike with Milikan).
However, the impression I have is that the vast majority of challenges are strictly along tribal lines – that very few people are making much effort to be objective. And of course, the forces at play are not in any way equal. The challenge to the hockey Stick was carried out not by mainstream climate scientists [why would they challenge one of their own, even if they were riddled with doubts in private?], but by M&M in their own time.
And as has been mentioned on this thread a number of times, the subsequent attempts to replicate [cf validate] the Hockey Stick were beset with even more problems of bias than MBH, if that is possible.
I might sound like I think a vast conspiracy is going on – I don’t! I just think the unwitting biases in the particular subject we’re talking about are very strong. The emotions run high, there’s a lot a stake, and we have a topic where there are many degrees of freedom. Moreover, there is the whole issue of ‘translating’ tentative findings, for political and public consumption.
If you grant two things – that a) climate studies are are an area where [the many] degrees of freedom allow for researchers to extract statistical significance out of almost any data set.
and b) there are a large number of people already convinced of a particular phenomenon, who are motivated to confirm their beliefs
then I see a recipe for distortion. Not so much ‘false positives’, but exaggerations and unduly apocalyptic projections.
In your subject, I think that biases are mostly catered for – double blind trials etc. I don’t think climate science has found a mechanism yet to deal with its similarly inherent problems. At least not a very effective one.
I do think the Mann example is revealing here, Fred, because I don’t think there has been any correction by any of the team members and active attempts to keep opposing research out of the literature, particularly McIntyre’s work. In most fields, there would be some acknowledgment that maybe the previous work was questionable. The false positive I think was the idea that the current warm period was of a different magnitude than the previous 2000 years of climate history. This is I think still defended on Real Climate and in the recent statistical journals. So, that doesn’t qualify as “quickly” in my book.
I agree with Anteros’ post above my previous post. The way the Hockey Stick had to be challenged by outsiders is a symptom of a deep problem in the field. Perhaps it has to do with the fact that its a relatively small field compared to medicine where there are a lot more competing interests and points of view. I also believe that climate science is too dominated by some of the stronger personalities, such as Hansen and Trenberth. The attempts to exclude statisticians from climate studies are disturbing to me anyway. But climate scientists seem to almost be at war with statisticians. They aren’t too comfortable with mathematicians either. Paul Williams is doing some really good work, but is still struggling to get the modelers to pay sufficient attention. I still don’t understand why Hansen doesn’t include people from these fields in his team and have them try to blow holes in his models. Either you do it yourself, or you will look bad later if you turn out to be wrong.
Actually we don’t know, whether the hockey stick had to be challenged by outsiders. I know that at least some main stream climate scientists didn’t give much weight at the evidence at any point (my fully certain case is from Finland, but I think the sama attitude was common).
We’ll never know, what would have happened without the critique of McI and McK, as their entry changed the stage. This has been one of the main worries that I have with the polarized discussion. The scientists became less open than they normally are. They did hide their critique more than is normal and didn’t bring it to public as much.
Some scientists are always following the dominating paradigm. That may make their path smoother. At the same time all best and self confident scientists have exactly the opposite goal. They wish to find something that contradicts the prevailing understanding. It’s not possible the climate scientists would be different in that, but there may be peer pressure that keeps them more quiet about their views.
I find the discussion of Mann’s work amusing. Pekka Pirilä says:
I suppose in some sense, “we don’t know” that. However, it’s a reasonable supposition. Even now, the IPCC AR5 ZOD includes both MBH and Mann 2008. That’s right. Two completely flawed papers are being currently being included in the latest IPCC report despite their flaws being well known.
Is it possible the hockey stick didn’t need to be “challenged by outsiders”? Sure. Maybe if nobody had pointed out how wrong it was, people would have stopped using it. Maybe the only reason it is still being included in IPCC reports is that people pointed out how flawed it was. I find it unlikely though.
But it is a funny idea. Can you imagine how twisted the IPCC process would have to be that pointing out flaws in the work it depends on makes it promote that work even more than saying nothing? I think that’d be more damning than the alternative.
“The false positive I think was the idea that the current warm period was of a different magnitude than the previous 2000 years of climate history.”
David – If that’s what you believe, then I have to say you are wrong and that you missed the point of my earlier comment. The “idea” you refer to is not a false positive – it’s a positive in dispute, with many studies supporting the idea and others opposing it (I would add that it’s also less important than warranted by the degree of argument it has generated, so it would be a mistake in my view to reargue it here once again). The only “positive” that is clearly false at this point rather than unsettled is the claim that the method used by Mann et al demonstrated the truth of that idea with high confidence. The false aspect lay in the methodology, not the conclusion, and I believe there is now widespread agreement on that point.
Paul Williams, a Royal Society University Research Fellow in climate modelling at the University of Reading, UK, agrees that the model is a useful tool. “Even the most hardened climate sceptic with a basic knowledge of physics could not possibly object to the application of energy conservation to the climate problem,” he says. “The energy-balance method provides further independent evidence for the anthropogenic origin of the majority of 20th-century global climate change.” …
I believe this is the model Fred Moolten was being slammed for defending.
Yeah, let’s get Hansen and Williams together. Who knows, in a day or two they might get the oceans to boilin’! That is, in few thousands years.
Fred, I’ve looked into this in some detail and it seems to me that most proxy studies show a range of variations over the last 2000 years that is order 1-2 degrees K. It is only the use of “Mike’s Nature trick” that enables the claim about the recent warming being unusual. I’d be open to further evidence but I note that Skeptical Science still uses this trick in a recent post. Even then, you have to agree that proxies have a lot of problems. I think the historical evidence is probably better.
In any case, even assuming that this issue has not been settled, one of the two views, the historical view of the Mideval Warm Period, or the Mannomatic view will ultimately become a false positive. This is far from your assertion that false positives become acknowledged “quickly.” It is not hard to see why since the data is noisy and people have political agendas here. A lot of people I respect say so, such as Muller, Curry, Lindzen, etc. As we have discussed before, this issue may be more a matter of personality. Some people have higher standards of integrity than others.
I also think that what you describe as the settled view that the methodology was flawed but that the conclusion might be right is not clear to any outsider looking at this field. I would say that at least publicly, I have seen no team members say that the methodology was flawed. Certainly, I’ve seen none say that the conclusion might be wrong. That’s a problem I think with the literature here. You can’t trust it on such matters. McShane And Wyner appeared in a statistics journal, not a climate journal.
JCH, It would be very helpful for Williams to spend some time working on the GISS model and doing rigorous numerical studies. I’m not sure about his statements on conservation of energy principles. We’ve been through that before and its an issue that I would say is “settled science” in other fields, namely, that global conservation of energy is a weak constraint on dynamics and thus tells you little. That’s not universally true, but its clear to anyone with any experience modeling fluid flows that its a weak constraint generally.
David – The points I want to make can be read by anyone who reviews my earlier comments. Probably the only one worth reiterating is that in a scientific arena where claims are quickly challenged, false positives are likely to be short lived. Incorrect conclusions may remain in dispute for a while, but their tenure as established, generally accepted positives will be limited, even if their originators and some defenders continue to assert them. Concepts that remain widely agreed upon for long periods despite being wrong will be relatively few under these conditions.
I’ll leave that as a generalization, but I haven’t seen contradictory evidence, nor important contradictory examples at this point. I’ll probably refrain from further commentary unless something new is introduced into the discussion.
DY – you probably correct as he appears to be a very bright guy who understand humans have very likely caused most of the warming since 1950, but based upon what he is saying GISS Model E is not one I think he would choose as a productive use of his time.
Fred, I think we are going to have to agree to disagree. You have, it seems to me, more confidence in the process of scholarship and science than I do. My final point would be that those who err on the side of skepticism are more likely to make big contributions, especially if they are persistent. Those who pay a lot of attention to their peers will make contributions too, but of a more evolutionary kind. So, in a way, I would be disappointed if I totally convinced you and vice versa. Both types are needed.
Judith,
A manipulation of single mathematical calculation is not what an orb and it’s complexity is.
Funny how we worry about CO2 and not the heat we generate to create CO2….
Dr. Curry,
Since False Positives seems to bring out the entertaining side of blog science, how about an apples and oranges post?
I see plenty of apples and oranges comparisons. The most common is RMS to Peak-Peak, smoothed paleo reconstructions are essentially RMS values of temperature and instrumental records are closer to peak-peak values.
Black swans, like a rogue waves or the 1997/8 super el nino are peak-peak phenomena, in my opinion, being compared to milquetoast 70 to 100 year averaged RMS values.
This may just be my perspective, but it appears to be a major issue.
if we get a super el nino in 2013-2014 even bigger than 1998, like Robert has predicted (I may be off a bit on the year), then what is it?
It will be another large El Nino event.
Likely another shift. Not necessarily a shift up either. I am trying to find some papers on pattern recognition of chaotic events. The super El Nino peak-peak has to be classic and the 1994 squiggle “w” appears to be classic, but was missed by Douglass. Interesting, kinda like scientific tea leaf reading :)
BTW – Cap’n
I think you might like this:
Steve McI has referred to it as tea leaves, also entrails. Fish guts?
Based upon recent behavior, to set a record all that is required is neutral conditions with a prediction of probable in the El Nino. It does not have to be an especially strong El Nino. A snotty little stupid one about the size of Joshua will do the trick.
Bacon Latte? A bacon latte with a baco-taco and a side of my bacon cornbread muffins, that sounds like a breakfast special!
http://www.google.com/imgres?imgurl=http://laist.com/upload/2010/09/lardon-baco.jpg&imgrefurl=http://mobile.laist.com/2010/09/21/fo_sizzle_meet_the_all-bacon_lardon.php%3Fgallery0Pic%3D1&h=414&w=640&sz=58&tbnid=otyfxqKX4Oh-ZM:&tbnh=90&tbnw=139&zoom=1&docid=duvpf0ClWGaGVM&sa=X&ei=plYQT4byLc3Mtgfgt9n9AQ&ved=0CEgQ9QEwBg&dur=1225
http://www.motifake.com/image/demotivational-poster/1108/death-bacon-pig-death-demotivational-posters-1314697859.jpg
Until the time it becomes fashionable to submit scientific results to ‘The Journal of Negative Results’, I suspect that we are stuck with a high level of false positives. Especially when it is so easy to perform ‘experiments’ on a computer.
In areas most closely related to my own discipline, I get a certain level of enjoyment from being able to spot a clearly ‘failed’ hypothesis where the results are presented as some other useful result [after application of a bit of lipstick]. I’ve also read that [properly regulated] clinical[drug trials are somewhat stricter about ‘panning-the-mud’ from a failed drug in the desperate attempt to rescue a lot of time, work, and money.
Hmmm.
False positives….
How about post after post based on the logic that because sometimes the “consensus” has been wrong, we should inherently question theories simply because they are supported by a “consensus?”
If you highlight a particular correlation, in contrast to a much stronger correlation (in this case the correlation between “consensus” and theories that have stood up over time and that we all freakin’ believe in and rely on, on a daily basis), without a scientifically validated assessment of causality – isn’t that essentially a false positive>
Interesting how some “skeptics” seem to be selectively concerned about false positives. A false positive wrapped up in a false positive, perhaps?
The smell of freshly mowed straw glibly fashioned into a quite hysterical faux argument. You never let us down, joshy.
Joshua,
False positives lead people to have unneeded surgery.
False positives get people indicted and jailed and convicted.
False positives lead to wasted resources.
But do tell us where skeptics here have suggested that questioning everything that is held in consensus is called for?
As Don points out, you seem to have a large supply of straw on hand.
Eli, made me do it.
“How about post after post based on the logic that because sometimes the “consensus” has been wrong, we should inherently question theories simply because they are supported by a “consensus?””
Joshua. The argument goes like this
AGW: Believe in the science, the consensus say so
Skeptic: The consensus has been wrong before.
That is different from the argument you suggest that people are making
Didnt you argue that your students had to learn how to present the other sides case?
you fail your own test
Right. “Skeptics” make bogus arguments because AGW-believers made them do so.
Now where have I seen that rationalization before?
Joshua,
You have yet to demonstrate what bogus arguments skeptics are making irt questioning consensus, nor why it is wrong to point out that in other similar situations where social movements have adopted sciencey arguments in a similar fashion to AGW things have gone poorly.
I am sure I will regret breaking my New year’s resolution so early.
hunter –
Looking past your selective process of determining “similarity” (a selective process of determining the definitional metrics to compare), there is nothing wrong, per se, with pointing out that consensus viewpoints are not always correct.
I mean it seems to me to be completely obvious, and so I wonder why some “skeptics” are so focused on pointing it out (kind of like when libertarians seem to think that they’ve had a brainstorm when they realize that actions often have unintended consequences), but there’s nothing inherently wrong with pointing it out.
The problem, IMO, is when, say, someone culls a list of instances when the consensus turned out to be incorrect from a massive list containing (arguably?) exponentially more examples where the consensus turned out to be correct.
IMO, that is the definition of a false positive.
Steve, If anyone is making this argument, I think (believe) we should ingnore them.
“AGW: Believe in the Science, the consensus says so.”
I think the most if not all AGW scientists are sayin something different, more on the lines of
AGW: Here is the science and the data, examine it for yourself, most scientists have done so and agree that the data supports the conclusions.
bob,
Your version is a fairer version than mine. so point taken.
In fact I think if the argument were framed differently the whole issue of “consensus” would not come up and the obvious skeptical counter would not be in play. The appeal to consensus and the appeal to “experts” seems to me to be rhetorically misguided. and counterplay against this strategy is so simple that a BOT could be programmed to do it. Long ago I suggested a different approach, one that targets the authority that many skeptics rely on. Simply, the core science of AGW, radiative theory, was advanced and developed into an engineering tool for the defense of this country.
The authority appealed to here is not some ivory towered professor ( a cartoon my right wing friends tend to dislike) but rather the crusty old engineering wizards. The appeal is to what works, rather than papers and models and theories. That’s also a cartoon, but you have to know your audience. Trying to appeal to the right side of the political spectrum with the appeals to experts and appeals to consensus is just wrong headed.
I would guess they never talked to a conservative when crafting their arguments. They did not understand their audience, they still dont get their audience. They wont listen to constructive criticism. They keep repeating the same approach and expecting different results.
Josh,
You are making gibberish. You are simply trying to turn the tables and avoid the issue by redefining the definition of false positive. I guess it is because you have nothing relevant to offer.
maybe you should go have a conversation with yourself
http://www.cleverbot.com/cleverthem
Why?
It’s more fun to post comments and watch you obsessively respond (with visions through a “window into [my] soul,” and then express outrage at how it’s my fault that you’re distracted from your important work.
You know, because it’s my fault that you respond to my posts.
Same old, same old.
Seems like it’s one argument fits all sizes for you, eh?
Steven,
You are obsessing. And you are an inveterate attention seeking little twit. On every thread here, you try to hijack the conversation. You insult women, Republicans, Christians, and just about everybody else, except bobbie.
You never have any specific comment about the science, cause you don’t know squat about it. In fact, you don’t seem to know squat about anything in particular, except circular argument. You obviously have some kind of training in progressive babbling. Did you go to one of those expensive little toney left-wing Liberal Arts colleges? BA in Polemics? MA in Semantic Quibbling, or Obfuscation?
Oh, sorry Steven. This was obviously meant for somebody else.
Don,
He still doesnt get it. Pretty funny.
oh my Don, you are distracting me from my important work.
Opps compile is done, BRB.
So how does this make kim a bot?
Never mind – I think I’ll stick with my fantasies.
And sorry Josh, mine don’t involve you.
Steven,
I bet I had him going for a while. As he read the first few sentences, he was thinking “OMG, he is slapping Mosher around this time. Go donnieboy!” But as he got further along it started to dawn on the little twit that nothing had changed.
He is really pathetic. He thinks Judith just says she doesn’t take him seriously to avoid being taken down by his witty and charming arguments. But she really doesn’t take the little boy seriously, at all. Nobody does. Even his nominal ally, Fred, studiously avoids getting involved in his mendacious meanderings.
Now enough fun. Get back to work.
Mosher and Don – best buddies, who’d of thought it?
sort of shows Mosher’s true colours
Are you new here Lousie? Mosher and I have had our differences, but as I came to realize that he knew a lot more about the science than I do, I decided to learn from him. I have moved closer to his position. It’s called keeping an open mind, and using one’s head. That and our mutual appreciation of Dr. Funkenstein and the P-Funk:
Louise –
It seems that their similar fantasies create a strong bond.
Both Mosher and I see you for what you are. And we ain’t the only ones. Nobody likes you. And it’s got nothing to do with your sexual orientation, so don’t bring that up again. Check yourself, joshy. Try to be a better man/person?
First, we know that GCMs are very inaccurate tools even when it comes to predicting the weather. Predictions more than a few weeks out can vary dramatically from forecaster to forecaster.
If you plan a wedding based on a very positive forecast in February about there being a dry Sunday during the third week of April, would the odds of a “false positive” devastate your future?
And, if you know the odds of a “false positive” are alarmingly high will you continue to invest thousands of man hours and millions of dollars improving predictions and making potentially unnecessary contingency plans for a devastating and debilitating rainstorm based on the uncertainty of weather predictions, especially you realize that the chances of an especially devastating rainstorm is very rare and worrying about would be largely unnecessary because there is nothing you can do about anyway?
When it comes right down to it, continuing to spend more and more money on more and more filing cabinets of worthless junk science is not the way to go if you want a strong and growing economy and a bright future for your children and the country.
Let’s be clear why the false positives problem is a problem for AGW research. Note that I have no objection to this research being published, as indeed it should be. But my impression is that this body of work, taken as a whole, would be regarded as run-of-the-mill had they been papers in other branches of the sciences. This is no insult: run-of-the-mill in good scientific journals is still deserving of respect. I say this because of the reliance on fitting complex statistical models to noisy data without a firm understanding of their reliability. This work should be a branching-out point for more research that either validates their claims, modifies them, or rejects them. This is how control over false positives is achieved. But instead of allowing this process to run its course, the proponents of this research seized the opportunity to shut down the game by taking it to the political arena and proclaiming that the science is settled. Now the only control over false positives is what a small collection of personalities decides that it wants to allow.
Obviously, under certain circumstances, raw data WILL be re-analyzed. This may only happen in fields/experiments which could provide an impetus for high cost changes in society, such as climate science. These low probability/high consequence scenarios are compelling enough to make raw data available for all published papers. The cost of this is low given the internet and the payoff to the public could be very high.
One of the problems of computerized models is that they are computerized models. Now almost all computer languages are definitive and strict in the relationships they define. Logical and mathematical by nature they require that all variables be explicitly defined as to certain very limited mathematical or logical qualities and all relationships between variable be explicitly set out. That requires an intimate understanding of the phenomena you are trying to model. An understanding that in complex subjects such as psychology or climate science, no one can reasonably claim. The result is that the researcher is forced to radically simplify the variables and logical structure in an effort to create a model that can be implemented in computer code. The result is as A. Chapanis said in his presidential address “My final criticism of model building, is that the modeler often becomes so intrigued with the formulation of his models that he constructs them for essentially trivial problems. Having at one’s disposal a large electronic computing machine, for example, invites one to try out all kinds of things, because computers are such fun to play with. Considering the state of knowledge within psychology, however, the easiest problems to build models for are essentially unimportant problems.” This is quoted in Muzafer Sherif’s interesting discussion of social psychological effects on theoretics and research. See http://www.brocku.ca/MeadProject/Sherif/Sherif_1966c.html.
I think Climate Science has succumbed to the great mythos of computers, that most scientific of machines. Only if you can codify your theories in computer code do you have a scientific theory. Unfortunately the immense limitations and constraints that they impose, are ignored in pursuit of the blessing of true science they are perceived to provide. If they are the most scientific of theories, than they must be the most grand containers of truth. And lest others realize that they tend to trivialize the phenomena one is trying to explain, they must be buttressed and defended without reprieve.
False(?) Positives
Example:
If in the 1940s some had projected continued warming, they would have been wrong as shown below.
http://bit.ly/zI5Sz6
It could be the same with the current continued warming projection.
CONGRATULATION TO ALL SKEPTICS.
A skeptic (David Whitehouse) has won a bet against a climate scientists (James Annan)!
http://bbc.in/wn7W2m
The bet was made in 2008, and it was whether the global mean temperature will exceed the previous record for 1998 by 2011.
It has not!
Again, congratulation to all skeptics.
http://bit.ly/xXLkUl
(source: http://bit.ly/zsFLFI)
I say the annul record of 0.548 deg C for 1998 from hadcrut3gl.txt will not be exceeded in the next three years, before 2015.
Any one wants a bet?
(we can contact “More or Less” of the BBC to organize the bet)
According to the Crackpot Index, you get 10 points for offering money.
Yet the BBC arranged, on behalf of an AGW promoter. to bet 100 pounds against a skeptic.
Dr. Curry,
Did I violate a posting rule by showing the link to the global climate tipping point video of Dr. Hansen speaking? I thought I posted it to this thread. If I violated any standard, please let me know and I hope you will accept my apology.
The problem isn’t so much that climate models are, of necessity, built on assumptions that are often untestable or untested or that, because of confirmation bias, there are many false positives.
I like Manacker’s idea that, when appropriate, authors and journal editors are required to state ”the results reported and the conclusions reached are not based on actual physical observations or reproducible experimentation, but instead on model simulations”. Confirmation bias is not a corruption of science, but the selling of model projections as scientific evidence is corruption. Hansen, Gore, and the IPCC, in their assessments, have done nothing to discourage journalists and laymen, who don’t know any better, from accepting model projections as evidence. For years now, people have heard and read that the science is settled, not realizing that, beyond atmospheric physics, little is known, but much is projected. That the Royal Society and Academies of Science (and their members) have allowed climate science to get away with this is a far greater problem than confirmation bias and false positives. It is corruption.
On the contrary, modeling results are a perfectly normal form of evidence. Modeling results are derivations from theory and to the extent that the theory is accepted these results carry a lot of weight. Most engineering is done with models, so there is no corruption per se just from using models. Solving a single equation is a simple case of using a model.
The issue is how good is the underlying theory and that is the core of the debate, with myriad sub-issues. Moreover, a lot of the theory built into the climate models is observationally based. I think the real issues with the climate models are those of omission, especially natural variability.
In any case it is just not as simple as you are making it out to be. Models are now central to all the sciences, and that is a good thing.
If some of the regular posters here want to have lively discussions with people who disagree with your perspective, I suggest they visit
http://scienceblogs.com/deltoid/2012/01/january_2012_open_thread.php
I would propose the term “Researcher degree of influence” rather than “Researcher degree of freedom”. It is more descriptive of the concern being discussed. It’s possible for non-experts to read the term “degree of freedom” and not immediately recognize it as a potential problem. This is because the term “freedom” isn’t specific enough in this context and also the plain english meaning is almost always positive (ie “freedom” is generally good). On the other hand, it would be clear, even to casual observers, that “influencing” the data is a potential problem (at least requiring proper disclosure and justification).
A very good point indeed. And the areas of influence are many and various – every choice right from research topic [does it have the potential to bolster the case for, say, AGW], through approach, colleagues, data, anaylsis, interpretation and so on. There is potential for ‘influence’ everywhere.
And you’re right – the word flags up ‘potential problem’ and the need for great openness and scrutiny, as self-scrutiny is such a rare commodity
Last year, I recall a respected psychology prof published a paper on evidence of ESP with statistically significant results (eg guess what card I’m holding, etc.). Many were speculating that it was something like “a Borat” to the research community, although the prof was not letting on if it was.
Does anyone else remember the specifics of this paper, and is there any connection with the paper being discussed?
If a .05 standard of statistical significance is used, it means that even with no ESP about 1 in 20 studies will produce a significant result. When that happens–time to publish and fatten up that CV! And just because a result is statistically significant doens’t necessarily mean that it’s important. If someone can correctly guess the outcomes of 5083 out of 10,000 coin tosses I am hardly impressed although it is statistically significant.
That’s a key point you make, Bob. I think there’s also not a few scientists/students who seem to think passing the ‘95% confidence’ level makes something ‘true’ in the sense of causation, not just correlation.
As an aside, I read recently that researchers looking for the Higgs Boson don’t consider something worthy of publication until it’s got past the 5 standard deviations benchmark.
You might want to read the following: Persi Diaconis, “Statistical Problems in ESP Research,” Science, vol. 201, 131-136, 1978. The author is a professor of statistics at Stanford. In his youth he joined the circus and did magic tricks.
I’ll check out, thanks,
However, just wanted to emphasize the idea that it was suspected to be satirical, although not explicitly stated, Since the prof was so respected, it was in fact detremental to his reputation to publish such claims, than beneficial to add another paper to his portfolio.
My philosophy prof 50 years ago, Paul Kurtz, has studied the claims of the paranormal all his life and is probably the best known American critic of paranormal claims. Philosophy of science was one of the most interesting topics I ever studied. Kurtz was the founder of CSI- http://en.wikipedia.org/wiki/Committee_for_Skeptical_Inquiry
I think this interesting article in Wired is relevant here and the “Too big to know” thread:
http://www.wired.com/magazine/2011/12/ff_causation/all/1
Thank Michael, this is a very interesting article
Michael, This is a fantastic article and I’m going to forward it to my brother. It is exactly along the lines of what he has been saying for quite a while. The statistical correlations have been getting worse and the cause and effect analyses much more tenuous. Generally, the evidence on lower back pain is that none of the invasive treatments seems to work much better than doing nothing. He has been cautioning me for 20 years about too strong a focus on cholesterol. Basically, your best correlation for how long you will live is how long your parents lived.
When testing new drugs in human clinical trials, the sponsoring company and the FDA meet before the trials begin to discuss that types of information the FDA would want to see before approving the proposed drug and what measure(s) of efficacy will define a successful treatment in what population of patients. During the trial, the sponsoring company may uncover alternative measures of efficacy or efficacy in a sub-population, but the FDA doesn’t allow the sponsor flexibility to move the goal post because it destroys the statistical integrity of the trial. If the first trial convinces the sponsor that an alternative patient population or measure of efficacy is more appropriate, they must spend the money to do a totally separate clinical trial.
An analogous process for climate science might proceed as follows. Let’s say that the IPCC hoped to claim in AR6 that it was very likely (p<0.1) that the upper tropical troposphere had been warming significantly faster than the near surface troposphere in the satellite era, just like GCMs predict. Scientists would have access to a small sample of the satellite data and some of the GCM data, but the bulk of the data would be withheld (say by NASA). NASA would convene a panel of climate scientists and statisticians to review the proposed analyses and individual scientists would show how their proposed analyses performed with the sample data. Acting much as the reviewers of a a scientific paper, the panel would approve or disapprove the methodology. Then the scientists would give program used to process the sample data to NASA, who would run the the approved program with the complete data. If the IPCC wanted to also address the same question on a longer timeframe using data from radiosondes a similar process could be used. The scientists would again receive only part of the radiosonde data and determine what methods, if any, should be used to homogenize the data or eliminate untrustworthy data and outliers. Their final product would again be a program (ie, no subjectivity allowed) that would process the bulk of the radiosonde data.
The FDA process must be followed before a drug company is allowed to sell $1B/year of a new drug. Shouldn't a similar process be followed for crucial elements (perhaps a dozen) of AGW before governments ask their citizens to spend trillions of dollars reducing greenhouse gases?
Yes. CO2 is a greenhouse gas.
However, the observed global temperature does not show accelerated warming with increase in CO2 emission.
Global Mean Temperature (GMT) data => http://bit.ly/pxXK4j
The most important observation in the above data is that the upper GMT boundary line passes through all the GMT peaks, the lower GMT boundary line passes through all the GMT valleys, and these lines are parallel. Also, the line that bisects the vertical space between the two GMT boundary lines is nearly identical to the long-term global warming trend line of 0.06 deg C per decade for the whole data. This result indicates that, for the last 130 years, the GMT behaved like a stable pendulum with the two GMT boundary lines that are 0.5 deg C apart as the end points of the pendulum’s swings, and the long-term global warming trend line of 0.06 deg C per decade as the pendulum’s neutral position.
In the above data, the GMT touched its upper boundary line only 3-times, about every 60-years, but has never crossed it for long in the last 130 years.
In the GMT data, a shift in climate to an accelerated global warming would have been indicated if the upper GMT boundary line had been a curve with an increasing positive slope with increasing years, or the upper and lower GMT boundary lines had been diverging with increasing years.
Fortunately, the upper GMT boundary line is a straight line having, interestingly, the same global warming rate of 0.06 deg C per decade as the global warming trend line for the whole data. Also, the upper and lower GMT boundary lines are parallel, showing no change in the magnitude of the GMT swing with increasing years. As a result, the vertical cooling or warming swing of 0.5 deg C between the two GMT boundary lines is cyclic and is therefore natural.
However, there is evidence of a persistent but natural global warming of 0.06 deg C per decade. Not 0.2 deg C per decade as claimed by the IPCC.
False positives – the simple version : it’s simply more fun to find something than to find nothing.
Is degrees of freedom not just a euphemisn for lack of rigour?
The Traditional Egyptians used insulation to have their desert houses and structures awesome, as well as their linen clothing warmer within the cooler winter several weeks. http://7starspizza.myshowroom.se/about-us/ http://7starspizza.myshowroom.se/about-us/