by John Bates
A look behind the curtain at NOAA’s climate data center.
I read with great irony recently that scientists are “frantically copying U.S. Climate data, fearing it might vanish under Trump” (e.g., Washington Post 13 December 2016). As a climate scientist formerly responsible for NOAA’s climate archive, the most critical issue in archival of climate data is actually scientists who are unwilling to formally archive and document their data. I spent the last decade cajoling climate scientists to archive their data and fully document the datasets. I established a climate data records program that was awarded a U.S. Department of Commerce Gold Medal in 2014 for visionary work in the acquisition, production, and preservation of climate data records (CDRs), which accurately describe the Earth’s changing environment.
The most serious example of a climate scientist not archiving or documenting a critical climate dataset was the study of Tom Karl et al. 2015 (hereafter referred to as the Karl study or K15), purporting to show no ‘hiatus’ in global warming in the 2000s (Federal scientists say there never was any global warming “pause”). The study drew criticism from other climate scientists, who disagreed with K15’s conclusion about the ‘hiatus.’ (Making sense of the early-2000s warming slowdown). The paper also drew the attention of the Chairman of the House Science Committee, Representative Lamar Smith, who questioned the timing of the report, which was issued just prior to the Obama Administration’s Clean Power Plan submission to the Paris Climate Conference in 2015.
In the following sections, I provide the details of how Mr. Karl failed to disclose critical information to NOAA, Science Magazine, and Chairman Smith regarding the datasets used in K15. I have extensive documentation that provides independent verification of the story below. I also provide my suggestions for how we might keep such a flagrant manipulation of scientific integrity guidelines and scientific publication standards from happening in the future. Finally, I provide some links to examples of what well documented CDRs look like that readers might contrast and compare with what Mr. Karl has provided.
Background
In 2013, prior to the Karl study, the National Climatic Data Center [NCDC, now the NOAA National Centers for Environmental Information (NCEI)] had just adopted much improved processes for formal review of Climate Data Records, a process I formulated [link]. The land temperature dataset used in the Karl study had never been processed through the station adjustment software before, which led me to believe something was amiss. When I pressed the co-authors, they said they had decided not to archive the dataset, but did not defend the decision. One of the co-authors said there were ‘some decisions [he was] not happy with’. The data used in the K15 paper were only made available through a web site, not in digital form, and lacking proper versioning and any notice that they were research and not operational data. I was dumbstruck that Tom Karl, the NCEI Director in charge of NOAA’s climate data archive, would not follow the policy of his own Agency nor the guidelines in Science magazine for dataset archival and documentation.
I questioned another co-author about why they choose to use a 90% confidence threshold for evaluating the statistical significance of surface temperature trends, instead of the standard for significance of 95% — he also expressed reluctance and did not defend the decision. A NOAA NCEI supervisor remarked how it was eye-opening to watch Karl work the co-authors, mostly subtly but sometimes not, pushing choices to emphasize warming. Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets—in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.
Defining an Operational Climate Data Record
For nearly two decades, I’ve advocated that if climate datasets are to be used in important policy decisions, they must be fully documented, subject to software engineering management and improvement processes, and be discoverable and accessible to the public with rigorous information preservation standards. I was able to implement such policies, with the help of many colleagues, through the NOAA Climate Data Record policies (CDR) [link].
Once the CDR program was funded, beginning in 2007, I was able to put together a team and pursue my goals of operational processing of important climate data records emphasizing the processes required to transition research datasets into operations (known as R2O). Figure 1 summarizes the steps required to accomplish this transition in the key elements of software code, documentation, and data.
Figure 1. Research to operations transition process methodology from Bates et al. 2016.
Unfortunately, the NCDC/NCEI surface temperature processing group was split on whether to adopt this process, with scientist Dr. Thomas C. Peterson (a co-author on K15, now retired from NOAA) vigorously opposing it. Tom Karl never required the surface temperature group to use the rigor of the CDR methodology, although a document was prepared identifying what parts of the surface temperature processing had to be improved to qualify as an operational CDR.
Tom Karl liked the maturity matrix so much, he modified the matrix categories so that he could claim a number of NCEI products were “Examples of “Gold” standard NCEI Products (Data Set Maturity Matrix Model Level 6).” See his NCEI overview presentation all NCEI employees [ncei-overview-2015nov-2 ] were told to use, even though there had never been any maturity assessment of any of the products.
NCDC/NCEI surface temperature processing and archival
In the fall of 2012, the monthly temperature products issued by NCDC were incorrect for 3 months in a row [link]. As a result, the press releases and datasets had to be withdrawn and reissued. Dr. Mary Kicza, then the NESDIS Associate Administrator (the parent organization of NCDC/NCEI in NOAA), noted that these repeated errors reflected poorly on NOAA and required NCDC/NCEI to improve its software management processes so that such mistakes would be minimized in the future. Over the next several years, NCDC/NCEI had an incident report conducted to trace these errors and recommend corrective actions.
Following those and other recommendations, NCDN/NCEI began to implement new software management and process management procedures, adopting some of the elements of the CDR R2O process. In 2014 a NCDC/NCEI Science Council was formed to review new science activities and to review and approve new science products for operational release. A draft operational readiness review (ORR) was prepared and used for approval of all operational product releases, which was finalized and formally adopted in January 2015. Along with this process, a contractor who had worked at the CMMI Institute (CMMI, Capability Maturity Model Integration, is a software engineering process level improvement training and appraisal program) was hired to improve software processes, with a focus on improvement and code rejuvenation of the surface temperature processing code, in particular the GHCN-M dataset.
The first NCDC/NCEI surface temperature software to be put through this rejuvenation was the pairwise homogeneity adjustment portion of processing for the GHCN-Mv4 beta release of October 2015. The incident report had found that there were unidentified coding errors in the GHCN-M processing that caused unpredictable results and different results every time code was run.
The generic flow of data used in processing of the NCDC/NCEI global temperature product suite is shown schematically in Figure 2. There are three steps to the processing, and two of the three steps are done separately for the ocean versus land data. Step 1 is the compilation of observations either from ocean sources or land stations. Step 2 involves applying various adjustments to the data, including bias adjustments, and provides as output the adjusted and unadjusted data on a standard grid. Step 3 involves application of a spatial analysis technique (empirical orthogonal teleconnections, EOTs) to merge and smooth the ocean and land surface temperature fields and provide these merged fields as anomaly fields for ocean, land and global temperatures. This is the product used in K15. Rigorous ORR for each of these steps in the global temperature processing began at NCDC in early 2014.
Figure 2. Generic data flow for NCDC/NCEI surface temperature products.
In K15, the authors describe that the land surface air temperature dataset included the GHCN-M station data and also the new ISTI (Integrated Surface Temperature Initiative) data that was run through the then operational GHCN-M bias correction and gridding program (i.e., Step 2 of land air temperature processing in Figure 2). They further indicated that this processing and subsequent corrections were ‘essentially the same as those used in GHCN-Monthly version 3’. This may have been the case; however, doing so failed to follow the process that had been initiated to ensure the quality and integrity of datasets at NCDC/NCEI.
The GHCN-M V4 beta was put through an ORR in October 2015; the presentation made it clear that any GHCN-M version using the ISTI dataset should, and would, be called version 4. This is confirmed by parsing the file name actually used on the FTP site for the K15 dataset [link]; NOTE: placing a non-machine readable copy of a dataset on an FTP site does not constitute archiving a dataset). One file is named ‘box.12.adj.4.a.1.20150119’, where ‘adj’ indicates adjusted (passed through step 2 of the land processing) and ‘4.a.1’ means version 4 alpha run 1; the entire name indicating GHCN-M version 4a run 1. That is, the folks who did the processing for K15 and saved the file actually used the correct naming and versioning, but K15 did not disclose this. Clearly labeling the dataset would have indicated this was a highly experimental early GHCN-M version 4 run rather than a routine, operational update. As such, according to NOAA scientific integrity guidelines, it would have required a disclaimer not to use the dataset for routine monitoring.
In August 2014, in response to the continuing software problems with GHCNMv3.2.2 (version of August 2013), the NCDC Science Council was briefed about a proposal to subject the GHCNMv3 software, and particularly the pairwise homogeneity analysis portion, to a rigorous software rejuvenation effort to bring it up to CMMI level 2 standards and resolve the lingering software errors. All software has errors and it is not surprising there were some, but the magnitude of the problem was significant and a rigorous process of software improvement like the one proposed was needed. However, this effort was just beginning when the K15 paper was submitted, and so K15 must have used data with some experimental processing that combined aspects of V3 and V4 with known flaws. The GHCNMv3.X used in K15 did not go through any ORR process, and so what precisely was done is not documented. The ORR package for GHCNMv4 beta (in October 2015) uses the rejuvenated software and also includes two additional quality checks versus version 3.
Which version of the GHCN-M software K15 used is further confounded by the fact that GHCNMv3.3.0, the upgrade from version 3.2.2, only went through an ORR in April 2015 (i.e., after the K15 paper was submitted and revised). The GHCN-Mv3.3.0 ORR presentation demonstrated that the GHCN-M version changes between V3.2.2 and V3.3.0 had impacts on rankings of warmest years and trends. The data flow that was operational in June 2015 is shown in figure 3.
Figure 3. Data flow for surface temperature products described in K15 Science paper. Green indicates operational datasets having passed ORR and archived at time of publication. Red indicates experimental datasets never subject to ORR and never archived.
It is clear that the actual nearly-operational release of GHCN-Mv4 beta is significantly different from the version GHCNM3.X used in K15. Since the version GHCNM3.X never went through any ORR, the resulting dataset was also never archived, and it is virtually impossible to replicate the result in K15.
At the time of the publication of the K15, the final step in processing the NOAAGlobalTempV4 had been approved through an ORR, but not in the K15 configuration. It is significant that the current operational version of NOAAGlobalTempV4 uses GHCN-M V3.3.0 and does not include the ISTI dataset used in the Science paper. The K15 global merged dataset is also not archived nor is it available in machine-readable form. This is why the two boxes in figure 3 are colored red.
The lack of archival of the GHCN-M V3.X and the global merged product is also in violation of Science policy on making data available [link]. This policy states: “Climate data. Data should be archived in the NOAA climate repository or other public databases”. Did Karl et al. disclose to Science Magazine that they would not be following the NOAA archive policy, would not archive the data, and would only provide access to a non-machine readable version only on an FTP server?
For ocean temperatures, the ERSST version 4 is used in the K15 paper and represents a major update from the previous version. The bias correction procedure was changed and this resulted in different SST anomalies and different trends during the last 15+ years relative to ERSST version 3. ERSSTV4 beta, a pre-operational release, was briefed to the NCDC Science Council and approved on 30 September 2014.
The ORR for ERSSTV4, the operational release, took place in the NCDC Science Council on 15 January 2015. The ORR focused on process and questions about some of the controversial scientific choices made in the production of that dataset will be discussed in a separate post. The review went well and there was only one point of discussion on process. One slide in the presentation indicated that operational release was to be delayed to coincide with Karl et al. 2015 Science paper release. Several Science Council members objected to this, noting the K15 paper did not contain any further methodological information—all of that had already been published and thus there was no rationale to delay the dataset release. After discussion, the Science Council voted to approve the ERSSTv4 ORR and recommend immediate release.
The Science Council reported this recommendation to the NCDC Executive Council, the highest NCDC management board. In the NCDC Executive Council meeting, Tom Karl did not approve the release of ERSSTv4, noting that he wanted its release to coincide with the release of the next version of GHCNM (GHCNMv3.3.0) and NOAAGlobalTemp. Those products each went through an ORR at NCDC Science Council on 9 April 2015, and were used in operations in May. The ERSSTv4 dataset, however, was still not released. NCEI used these new analyses, including ERSSTv4, in its operational global analysis even though it was not being operationally archived. The operational version of ERSSTv4 was only released to the public following publication of the K15 paper. The withholding of the operational version of this important update came in the middle of a major ENSO event, thereby depriving the public of an important source of updated information, apparently for the sole purpose of Mr. Karl using the data in his paper before making the data available to the public.
So, in every aspect of the preparation and release of the datasets leading into K15, we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation. I finally decided to document what I had found using the climate data record maturity matrix approach. I did this and sent my concerns to the NCEI Science Council in early February 2016 and asked to be added to the agenda of an upcoming meeting. I was asked to turn my concerns into a more general presentation on requirements for publishing and archiving. Some on the Science Council, particularly the younger scientists, indicated they had not known of the Science requirement to archive data and were not aware of the open data movement. They promised to begin an archive request for the K15 datasets that were not archived; however I have not been able to confirm they have been archived. I later learned that the computer used to process the software had suffered a complete failure, leading to a tongue-in-cheek joke by some who had worked on it that the failure was deliberate to ensure the result could never be replicated.
Where do we go from here?
I have wrestled for a long time about what to do about this incident. I finally decided that there needs to be systemic change both in the operation of government data centers and in scientific publishing, and I have decided to become an advocate for such change. First, Congress should re-introduce and pass the OPEN Government Data Act. The Act states that federal datasets must be archived and made available in machine readable form, neither of which was done by K15. The Act was introduced in the last Congress and the Senate passed it unanimously in the lame duck session, but the House did not. This bodes well for re-introduction and passage in the new Congress.
However, the Act will be toothless without an enforcement mechanism. For that, there should be mandatory, independent certification of federal data centers. As I noted, the scientists working in the trenches would actually welcome this, as the problem has been one of upper management taking advantage of their position to thwart the existing executive orders and a lack of process adopted within Agencies at the upper levels. Only an independent, outside body can provide the needed oversight to ensure Agencies comply with the OPEN Government Data Act.
Similarly, scientific publishers have formed the Coalition on Publishing Data in the Earth and Space Sciences (COPDESS) with a signed statement of commitment to ensure open and documented datasets are part of the publication process. Unfortunately, they, too, lack any standard checklist that peer reviewers and editors can use to ensure the statement of commitment is actually enforced. In this case, and for assessing archives, I would advocate a metric such as the data maturity model that I and colleagues have developed. This model has now been adopted and adapted by several different groups, applied to hundreds of datasets across the geophysical sciences, and has been found useful for ensuring information preservation, discovery, and accessibility.
Finally, there needs to be a renewed effort by scientists and scientific societies to provide training and conduct more meetings on ethics. Ethics needs to be a regular topic at major scientific meetings, in graduate classrooms, and in continuing professional education. Respectful discussion of different points of view should be encouraged. Fortunately, there is initial progress to report here, as scientific societies are now coming to grips with the need for discussion of and guidelines for scientific ethics.
There is much to do in each of these areas. Although I have retired from the federal government, I have not retired from being a scientist. I now have the luxury of spending more time on these things that I am most passionate about. I also appreciate the opportunity to contribute to Climate Etc. and work with my colleague and friend Judy on these important issues.
Postlude
A couple of examples of how the public can find and use CDR operational products, and what is lacking in a non-operational and non-archived product
- NOAA CDR of total solar irradiance – this is the highest level quality. Start at web site – https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00828
Here you will see a fully documented CDR. At the top, we have the general description and how to cite the data. Then below, you have a set of tabs with extensive information. Click each tab to see how it’s done. Note, for example, that in ‘documentation’ you have choices to get the general documentation, processing documents including source code, data flow diagram, and the algorithm theoretical basis document ATBD which includes all the info about how the product is generated, and then associated resources. This also includes a permanent digital object identifier (doi) to point uniquely to this dataset.
- NOAA CDR of mean layer temperature – RSS – one generation behind in documentation but still quite good – https://www.ncdc.noaa.gov/cdr/fundamental/mean-layer-temperature-rss
Here on the left you will find the documents again that are required to pass the CDR operations and archival. Even though it’s a slight cut below TSI in example 1, a user has all they need to use and understand this.
- The Karl hiatus paper can be found on NCEI here – https://www.ncdc.noaa.gov/news/recent-global-surface-warming-hiatus
If you follow the quick link ‘Download the Data via FTP’ you go here – ftp://ftp.ncdc.noaa.gov/pub/data/scpub201506/
The contents of this FTP site were entered into the NCEI archive following my complaint to the NCEI Science Council. However, the artifacts for full archival of an operational CDR are not included, so this is not compliant with archival standards.
Biosketch:
John Bates received his Ph.D. in Meteorology from the University of Wisconsin-Madison in 1986. Post Ph.D., he spent his entire career at NOAA, until his retirement in 2016. He spent the last 14 years of his career at NOAA’s National Climatic Data Center (now NCEI) as a Principal Scientist, where he served as a Supervisory Meteorologist until 2012.
Dr. Bates’ technical expertise lies in atmospheric sciences, and his interests include satellite observations of the global water and energy cycle, air-sea interactions, and climate variability. His most highly cited papers are in observational studies of long term variability and trends in atmospheric water vapor and clouds.
NOAA Administrator’s Award 2004 for “outstanding administration and leadership in developing a new division to meet the challenges to NOAA in the area of climate applications related to remotely sensed data”. He was awarded a U.S. Department of Commerce Gold Medal in 2014 for visionary work in the acquisition, production, and preservation of climate data records (CDRs). He has held elected positions at the American Geophysical Union (AGU), including Member of the AGU Council and Member of the AGU Board. He has played a leadership role in data management for the AGU.
He is currently President of John Bates Consulting Inc., which puts his recent experience and leadership in data management to use in helping clients improve data management to improve their preservation, discovery, and exploitation of their and others data. He has developed and applied techniques for assessing both organizational and individual data management and applications. These techniques help identify how data can be managed more cost effectively and discovered and applied by more users.
David Rose in the Mail on Sunday
David Rose of the UK Mail on Sunday is working on a comprehensive expose of this issue [link].
Here are the comments that I provided to David Rose, some of which were included in his article:
Here is what I think the broader implications are. Following ClimateGate, I made a public plea for greater transparency in climate data sets, including documentation. In the U.S., John Bates has led the charge in developing these data standards and implementing them. So it is very disturbing to see the institution that is the main U.S. custodian of climate data treat this issue so cavalierly, violating its own policy. The other concern that I raised following ClimateGate was overconfidence and inadequate assessments of uncertainty. Large adjustments to the raw data, and substantial changes in successive data set versions, imply substantial uncertainties. The magnitude of these uncertainties influences how we interpret observed temperature trends, ‘warmest year’ claims, and how we interpret differences between observations and climate model simulations. I also raised concerns about bias; here we apparently see Tom Karl’s thumb on the scale in terms of the methodologies and procedures used in this publication.
Apart from the above issues, how much difference do these issues make to our overall understanding of global temperature change? All of the global surface temperature data sets employ NOAA’s GHCN land surface temperatures. The NASA GISS data set also employs the ERSST datasets for ocean surface temperatures. There are global surface temperature datasets, such as Berkeley Earth and HadCRUT that are relatively independent of the NOAA data sets, that agree qualitatively with the new NOAA data set. However, there remain large, unexplained regional discrepancies between the NOAA land surface temperatures and the raw data. Further, there are some very large uncertainties in ocean sea surface temperatures, even in recent decades. Efforts by the global numerical weather prediction centers to produce global reanalyses such as the European Copernicus effort is probably the best way forward for the most recent decades.
Regarding uncertainty, ‘warmest year’, etc. there is a good article in the WSJ: Change would be healthy at U.S. climate agencies (hockeyshtick has reproduced the full article).
I also found this recent essay in phys.org to be very germane: Certainty in complex scientific research an unachievable goal. Researchers do a good job of estimating the size of errors in measurements but underestimate chance of large errors.
Backstory
I have known John Bates for about 25 years, and he served on the Ph.D. committees of two of my graduate students. There is no one, anywhere, that is a greater champion for data integrity and transparency.
When I started Climate Etc., John was one of the few climate scientists that contacted me, sharing concerns about various ethical issues in our field.
Shortly after publication of K15, John and I began discussing our concerns about the paper. I encouraged him to come forward publicly with his concerns. Instead, he opted to try to work within the NOAA system to address the issues –to little effect. Upon his retirement from NOAA in November 2016, he decided to go public with his concerns.
He submitted an earlier, shorter version of this essay to the Washington Post, in response to the 13 December article (climate scientists frantically copying data). The WaPo rejected his op-ed, so he decided to publish at Climate Etc.
In the meantime, David Rose contacted me about a month ago, saying he would be in Atlanta covering a story about a person unjustly imprisoned [link]. He had an extra day in Atlanta, and wanted to get together. I told him I wasn’t in Atlanta, but put him in contact with John Bates. David Rose and his editor were excited about what John had to say.
I have to wonder how this would have played out if we had issued a press release in the U.S., or if this story was given to pretty much any U.S. journalist working for the mainstream media. Under the Obama administration, I suspect that it would have been very difficult for this story to get any traction. Under the Trump administration, I have every confidence that this will be investigated (but still not sure how the MSM will react).
Well, it will be interesting to see how this story evolves, and most importantly, what policies can be put in place to prevent something like this from happening again.
I will have another post on this topic in a few days.
Being retired sure is liberating . . .
Moderation note: As with all guest posts, please keep your comments civil and relevant.
Pingback: Climate scientists versus climate data – Enjeux énergies et environnement
Incredible ain’t it, non-archiving of critical evidence -?
Faith-based cli-sci- not for general exhibition.
“Incredible ain’t it, non-archiving of critical evidence -?”
And just not true. There is an extensive archive. Bates even linked to it. It is here.
Bates complaints seem to be
1. The archiving wasn’t complete until six months after the paper appeared
2. Data is in ascii format which is not “machine readable”. Of course it is, it just requires a format statement.
NIck you left out complaints:
1. Karl made administrative decisions contrary to data integrity;
2. Karl used 90% rather than 95% standard;
3. The use of non standardized data set implied a greater uncertainty to the data that was not, could not be addressed;
4. Karl made poor data integrity decisions in order to meet a publishing date;
5. Karl made decisions that benefited his publication not the organization;
6. etc., etc.
jfp,
“NIck you left out complaints:”
I dealt with the 90% issue here. He was comparing results with the AR5, which used 90%.
All the rest are matters of opinion, and his opinion does seem to be soured by something. 1,4,5 are pure opinion, and unsubstantiated. The non-standardized issue is bunk. If you go to the readme file in the archive, they say exactly what files they use:
“This directory contains the adjusted land station data and metadata used in the Old Analysis; data from GHCN-Monthly version 3.2.2.
ghcnm.tavg.v3.2.2.20150116.qca.dat.gz; Adjusted station data
ghcnm.tavg.v3.2.2.20150116.qca.inv.gz; Inventory of adjusted stations”
Those are standard issue, dated files. I download them every day. And the copies are in the archive.
And for the new Analysis
“This directory contains the adjusted land station data and metadata used in the New Analysis.
tavg.v4.a.1.20150119.qca.dat.gz; Adjusted station data
tavg.v4.a.1.20150119.qca.inv.gz; Inventory of adjusted stations”
This is V4, but again, standard dated versions, and are in the archive here
Nick, read this expose of John Bates claims:
http://icarus-maynooth.blogspot.com.au/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
More debunking here:
http://climatenexus.org/messaging-communication/current-events/climate-change-science-noaa-falsely-maligned-tabloid-spin#.WJfTMOTOsfs.twitter
Nick, yes to 1 and 2, leta not forget 3; where Karl is charged with deliberately manipulating data to spit out a very well timed lump of garbage and parade it around as observational science. Dont sugar coat what Karl did Nick.
” Karl is charged with deliberately manipulating data to spit out a very well timed lump of garbage and parade it around as observational science”
Sounds opinionated. And that is what we got from Bates. Just subjective, no supporting facts.
As Thorne (who was there), says, Bates was never involved at a technical level. All he has done is dig into the paperwork, and check whether boxes were ticked. There is a lot of ignorance in Rose’s article, at least some of which seems to originate with Bates.
“As Thorne (who was there)”
Evidently not, in fact.
Nick, yes i am opinionated and yes facts support my opinion regardarding fleet data.
Fact: a full survey and inspection of and not limited to, citing, materials, calibration, method, and procedural has never occurred relative to fleet measurements. Of all the variables affecting heat transfer and retention, all of these are unknown. The error bounds being as they are unknowable renders the data useless. That should be obvious to a child, youre not a child Nick. I can count raisins to see how many pounds of found my cat hungers for, but that’s not science Nick. Karls use of demonstrably corrupt “data” exposed him.
Owen
I would be interested in your opinion of a start date for reliable sst data. It relates to this question I asked nick
https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-836991
Tonyb
“Karls use of demonstrably corrupt “data” exposed him.”
The confusion of the complaints against Karl are amazing. K15 did not introduce the use of ship data. They have been used for very many years. Nor did K15 promote its use. On the contrary; the introduction of buoys, pioneered by NOAA, is replacing ship data, to an extent that, with variance weighting, it now provides by far the largest component of SST data. What K15 is doing is making possible a proper transition between the sources of data, by quantifying the (small) bias.
No, Bates states that (by definition) data are not archived by making it available somewhere as ftp. One of his major points is that Karl made public these data that had not been approved by formal process and want improvements so gained. Procedures like authority endorsement must surely be an initio taken as good and non- compliance, such as to time a paper’s release for impact, not so good. Agreed?
Geoff
Geoff,
“Procedures like authority endorsement must surely be an initio taken as good”
Ironies abound. Here is what skeptics had to say when someone deemed skeptic at CSIRO was reproved for completely ignoring publication approval processes.
But the stuff about “time a paper’s release for impact” is nonsense. Firstly, as ehak points out, it was actually submitted in December 2014. And I’m sure the dead weight of the Bates’s in NOAA would mean that they had it in process for long before that. But second, it is a methods paper. There is no particular reason why it should have impact, except skeptics decided for some reason that he was stealing their favorite pause. Basically, the information about measurable ship-buoy bias had been published for years, and ERSST just had to do something about it. That was the driver.
I followed the first link Ceist posted and behold – found this gem:
“v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. The assertion that buoy data were thrown away as made in the article is demonstrably incorrect.”
I was taught at an early age that you should never give people reason to believe you are a liar, a thief or a cheat. Well, the author of that post gives plenty of reason to believe he is the latter with this statement.
Sure the buoy data was weighted heavily – AFTER it was adjusted against the ship intake data. Now I’m not taking issue with that decision, even though it sounds counter intuitive to me – taking the less reliable data set and adjusting the more reliable one to it. What I do take issue with is the author of the post spinning it to make it sound as if Dr. Bates is making a false statement. To claim that “v4 … makes preferential use of buoys over ships ” is effectively an effort to misdirect, as no mention is made of adjustments to it prior to it being used. And if I am recalling correctly, Dr. Bates didn’t say the unadjusted buoy data was “thrown away”, rather that it was not archived in the standard manner.
tim –
==} And if I am recalling correctly, Dr. Bates didn’t say the unadjusted buoy data was “thrown away”, {==
–snip–
Dr Bates said: ‘They had good data from buoys. And they threw it out and “corrected” it by using the bad data from ships. You never change good data to agree with bad, but that’s what they did – so as to make it look as if the sea was warmer.’
–snip–
http://www.dailymail.co.uk/sciencetech/article-4192182/World-leaders-duped-manipulated-global-warming-data.html#ixzz4XwLUFctN
“The confusion of the complaints against Karl are amazing. K15 did not introduce the use of ship data. They have been used for very many years. Nor did K15 promote its use. On the contrary; the introduction of buoys, pioneered by NOAA, is replacing ship data, to an extent that, with variance weighting, it now provides by far the largest component of SST data. What K15 is doing is making possible a proper transition between the sources of data, by quantifying the (small) bias.”
I can’t speak for others Nick, but I’m not operating under any sense of confusion. And you are the one introducing the strawman with the above comment. Who is claiming Karl “introduced” the use of ship intake data? No one I’ve seen. And the term “promote” is another effort at misdirection. The primary question regarding the use of the two data sets was towards the logic behind how they were utilized. Why adjust the more reliable set using the set having lower reliability and then weighing the adjusted set more heavily? I have yet to see a good answer to that simple question. Care to provide one or are you too busy closing ranks?
Probably time to start thinking about another line of work, Nick.
Nick kindly left out the bit about Matrix maturity model.. pieces of gold eh, except the data was never assessed to earn such a rating.
Cherry pick fail Nick, you seem to do that a lot.
Lean startup fail fast
“There is no particular reason why it should have impact, except skeptics decided for some reason that he was stealing their favorite pause. ”
Nick, this claim is just bizarre. Are you really unaware of the massive media coverage around this study? The acompanying PR blitz by the usual promoters of global warming fears?
That climatophobes can spend a year loudly trumpeting a result in major media and then argue the result was only interesting because of the reaction of skeptics really speaks to the state of the movement.
One thing that is ‘incredible’ is David Rose’s dishonest graph:
https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise
The bias and Activism in NOAA is what is incredible.
Exactly what is “dishonest” about which “graph” (there are six graphs in the linked article)?
The last graph (“Hadley and NOAA – Common Baseline”), when El Niño (geothermal) distortions are removed (or just look at 2001 – 2015) appears to confirm the hiatus. Are you suggesting the graphic does not represent Hadley and NOAA data?
David Rose and John Bates exposed here:
http://icarus-maynooth.blogspot.com.au/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
More debunking here:
https://www.theguardian.com/environment/climate-consensus-97-per-cent/2017/feb/05/mail-on-sunday-launches-the-first-salvo-in-the-latest-war-against-climate-scientists
Jeremy Berg, Editor of Science:
“With regard to the “rush” to publish, as of 2013, the median time from submission to online publication by Science was 109 days, or less than four months. The article by Karl et al. underwent handling and review for almost six months. Any suggestion that the review of this paper was “rushed” is baseless and without merit. Science stands behind its handling of this paper, which underwent particularly rigorous peer review.”
Jane Lubchenco, former NOAA administrator:
“These are sad, old accusations that have been definitively disproven.”
Rt. Rear Admiral David Titley, former NOAA chief operating officer:
“In summary, the Mail on Sunday has found a disgruntled ex-NOAA employee and is using him to construct alternative facts about the climate. Unfortunately for all of us, the air will keep warming, the seas will keep rising, and the ice will keep melting, regardless of the Daily Mail’s fanciful claims and accusations. The real atmosphere is impervious to alternative facts.
There is both a NOAA internal process on scientific integrity (my office ran it when I was at NOAA) and the opportunity to submit allegations of wrongdoing to the Department of Commerce Inspector General who, if there is reasonable evidence to substantiate the allegation, would undertake an independent investigation”
Ceist,
learn to read, as with Admiral Titley’s comment:
“In summary, the Mail on Sunday has found a disgruntled ex-NOAA employee and is using him to construct alternative facts about the climate. Unfortunately for all of us, the air will keep warming, the seas will keep rising, and the ice will keep melting, regardless of the Daily Mail’s fanciful claims and accusations. The real atmosphere is impervious to alternative facts.”
I didn’t bother reading the Mail article, just Dr. Bates post above. Not once does he “construct alternate facts about the climate”. He talks about process. And are you really going to give weight to any individual who claims “Unfortunately for all of us, the air will keep warming, the seas will keep rising, and the ice will keep melting .. “? That is most certainly not a scientific statement. It is a play on the emotions, with no proof to support the unfortunately part. In fact the science we have tells us exactly the opposite when it comes to ” the ice will keep melting”. Our planet has spent far more time in periods of large scale glaciation then is has in interglacial periods like the current one over the past million years or so. Based on our knowledge of the past record we should be coming to the end of the current period. If that is the case, I’m thinking a bit of warming is a good thing. Particularly was no one has shown how it will be bad.
Neither of those debunk anything. Strange
The link was interesting but only in an ironic “Nixon staffer debunks Watergate” fashion. Honestly, what did you expect them to say?
Incredible ain’t it? Why hasn’t there been a big stink about Spencer and Christy’s UAH tropospheric satellite dataset? It’s still beta version 6.x and has been used for almost 18months now. Their paper explaining the underlying changes has still not been published.
RSS updated their satellite data set to Version 4 last year. They waited until after their paper has been published to release the data set. The old Version 3 (which is closer to UAH 6.x beta) runs colder because of various issues outlined in their paper.
Yet Judith Curry, Ted Cruz etc run around claiming “The satellite data is the best data we’ve got!” (referring to UAH). Despite the satellite data having much higher error range than land based temperature data.
Remind me again what public policy is dependent on ignored by IPCC Satellite data sets?
Isn’t all of the data… corrected? My understanding is that the world is divided up into grids with official thermometers in them and when we look at the data, only 20% of these grid cells actually have official thermometer readings. That’s a lot of missing data that — gaps in the record — for which “missing data” is simply provided tp fill in the gaps where no data exists or has been lost or deleted or has otherwise simply gone missing.
My understanding is that the missing data is filled in from the computer models (the ones that run too hot). If you were wondering why the official NOAA and GISS temperature plots look like the models, it’s because they threw out any records that were too cold and inserted model warming.
You can call it scientific fraud if you like. NOAA and GISS justify it by showing the nice hockey stick they’ve made.
“missing data” is not filled in with “computer models.”
For Land NOAA and GISS use IDW.
For SST EOF and then Gridded
Both of these are established methodologies for doing spatial prediction
otherwise known as interpolation.
Tested and verified
“Both of these are established methodologies for doing spatial prediction
otherwise known as interpolation” – phew, good thing you’re not using “computer models” to fill in the data. Oh, wait. Huh. You’re interpolation to estimate data between known values… yeah, a model. And I bet you are using a computer to do it.
There are statistical models
physical models
niave models
all sorts of models
You can compute them by hand, in your head or on a computer.
Mosh you cant verify data you dont have.. mind you its not as bad as GISS infilling
The world is not divided into Grids.
NOAA uses EOF for SST, gridding is just a simplified means of distributing.
Same goes for berkeley land surface. We have a continuous surface. No grid. For output we choose 1 degree, we;vee also done 1/4 degree and
long ago 30 arc seconds for Google earth
A continuous temperature “surface” based on a sparse convenience sample uses infinitely more interpolation than a gridded area averaging method. But neither should be used to draw statistical conclusions about the total population, in this case global temperatures and the global average thereof.
Statistical sampling theory is perfectly clear about this inadequacy, but it is universally ignored by the statistical modelers. K15 is bad but the whole surface temperature game is a statistical sham.
Question: how much of the “data” that is output when you choose 1/4 degree is an estimate of unknown points from known points? One percent? Two percent?
Sure, sure, averaging averages… that’s the ticket– based on such results, I probably can pack a lot lighter this year for my trip to Antarctica.
–e.g.,
David, you shouldn’t do a lot of things if you want the most accuracy in life, but can you quantify the failure as you see it a little bit more precisely? If we have limited samples, do you suggest we ignore the issue altogether? Can you write up a paper showing the sort of error being introduced that allegedly is not already being accounted? If the end result is a small deviation, all your complaining would have amounted to little more than unwarranted skepticism. If it’s significant, you might get a major science prize or something. What do you say?
Hydrodynamic fiinite element models have evidently evolved a lot faster than Steve’s comprehension of them, and what they can do.
An adjustment too far :-D
The Window To Hell is slowly opening. What a way to do science! Apparently there is a discipline called a “political science”. I suggest that Climatology should be merged with it.
I would agree if we were only talking about unconscious ideologically-motivated bias but as we learned as early as the foi2009.pdf-CRUgate disclosures, we also dealing with overt dishonesty.
Scientists should be required to work to engineering standards for archiving data.
It’s always amazed me how the organizations have got away without proper quality controls. Are they ever externally audited? My company was regularly audited by two external organizations.
Steven Mosher:
You touch on – but evade – an important issue when you write,
“There are no established, globally accepted, universal “engineering standards”.
There is no possibility of such “standards” because
(a) there is no “established, globally accepted, universal” definition of global temperature
and
(b) if there were such an agreed definition then there is no possibility of an independent calibration standard for it.
Indeed, each team that provides global and hemispheric temperature data sets alters the definition it uses almost every month and, therefore, changes its data for past global temperatures almost every month; see e.g. http://jonova.s3.amazonaws.com/graphs/giss/hansen-giss-1940-1980.gif
All supposed “data” for global temperature is rendered scientifically invalid by the lack of an agreed definition of the parameter and the impossibility of any calibration standard for it. Simply, estimates of global temperature are pure pseudoscience with less credibility than phrenology.
Richard
There are no established, globally accepted, universal “engineering standards”
Suggest a CONCRETE example of a written standard and you have an argument.
Like an ISO standard
Mosher,
You dodged the important question I asked you upthread: https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-836575
Avoiding it suggest you recognise there is no better chart of temperature over the Phanerozoic Eon, despite 30 years of intensive climate research, and you recognize temperatures have been far higher for most of the past 542 Ma, life thrived, and there is no valid evidence to support your beliefs that GHG emissions are net-damaging nor that 2 C warming is dangerous. (Explained here: https://judithcurry.com/2017/01/29/the-threat-of-climate-change/#comment-836115 )
This is the Achilles Heel of the Alarmists’ belief.
ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories is the main ISO standard used by testing and calibration laboratories. In most major countries, ISO/IEC 17025 is the standard for which most labs must hold accreditation in order to be deemed technically competent. In many cases, suppliers and regulatory authorities will not accept test or calibration results from a lab that is not accredited. Originally known as ISO/IEC Guide 25, ISO/IEC 17025 was initially issued by the International Organization for Standardization in 1999.
The Joint Committee for Guides in Metrology (JCGM), is an organization that prepared the “Guide to the expression of uncertainty in measurement” (GUM) and the “International vocabulary of metrology – basic and general concepts and associated terms” (VIM). The JCGM assumed responsibility for these two documents from the ISO Technical Advisory Group 4 (TAG4).
These two standards are freely available at Bureau International de Poids et Mesures
http://www.bipm.org/en/publications/guides/
This might be a duplicate. Posting from an iPad for the first time. Anyway. NI-43-101 is a globally accepted, established and enforced standard. Karl would be charged under securities law if he were a geologist and did what he did.
Peter
Sorry I did not dodge ANY important question you had.
Youve never asked an important question in your life
How many times do people have to refuse to change your damn diapers for you.
Ps.. the chart you provided is crap.
Mosher,
Why is the chart crap? Is thee a better chart? Why is it better?
I suspect there is not, because neither you nor anyone else has provided a better one, including RealClimate.
Mosher dodge the question here. He replied down thread (to one of his own comments):
Mosher gets really upset every time he is asked for evidence that global warming is net-damaging, serious threat, dangerous. He can’t answer the questions with a constructive, informative answer, and instead flies into an abusive rage. He says the chart is crap, but didn’t provide a link to a better one and explain why it is better. He avoided the question: Sign 4 of <"10 signs of intellectual dishonesty":
https://judithcurry.com/2013/04/20/10-signs-of-intellectual-honesty/
Huh, no established, globally accepted, universal “engineering standards”? I work in the “evil oil industry” and the entire industry runs on the API or American Petroleum Institute standards, from wellhead to well casing to drilling fluids. The API engineering standards as globally known and serve as the benchmark. Official API documents are divided into specific Standards as well as “Recommended Practices” to cover operational procedures. The US Government incorporates many of these Engineering Standards into the Federal regulations as do many other Government Regulators all over the world.. To say there are no established, globally accepted, universal “engineering standards” is simply ignorance.
Aren’t you the fella who doesn’t accept the greenhouse effect?
No!
Isn’t this your blog Peter Ward?
http://ozonedepletiontheory.info/index.html
Or David Lang? Apologies if it’s not yours.
I wouldnt cast stones if i were u. Your blinders are so high up your face your eyes are bruised. Thomas Karl used a ridiculous dataset, and magically it found its way into circulation. Magic to you perhaps. To the rest of us it looks real bad. His head should be on the proverbial pike right now and you defend him by attacking the whistleblower. Manage yourself with more integrity.
No Ciest. If you want to know about me read Denizens II (top right of the CE page).
Short bio here: https://anglejournal.com/article/2015-11-why-carbon-pricing-will-not-succeed/
Making wild, baseless assertions is a sign of intellectual dishonesty. I’d suggest you are displaying that in spades.
How embarrassing to discover you are Australian: https://twitter.com/Ceist8
Since you seem to like guessing, I’ll take a guess that a description of you: You are probably one of those gullible young dudes who accept the nonsense sprouted by the likes of John Cook (SkepticalScience) and his disciples. In capable of challenging your beliefs and doing your own reality checks. Furthermore, you hide behind a pseudonym and are not prepared to state your real name or your background.
Peter, having read your “I am persuaded… ” statements, you seem very easily persuaded by anything you believe supports what you want to believe. It looks more like you are projecting and that you yourself are “incapable of challenging your beliefs and doing your own reality checks”. I will say back to you “How embarrassing that you are Australian”.
….But at least you aren’t Peter Ward.
Geeze Ceist, you can’t get something as simple as Peter Lang’s name right?
Like… Spencer and Christy’s UAH tropospheric satellite dataset? It’s still beta version 6.x and has been used for almost 18months now. Their paper explaining the underlying changes has still not been published.
RSS updated their satellite data set to Version 4 last year. They waited until after their paper has been published to release the data set. The old Version 3 (which is closer to UAH 6.x beta) runs colder because of various issues outlined in their paper.
Yet Judith Curry, Ted Cruz etc run around claiming “The satellite data is the best data we’ve got!” (referring to UAH). Despite the satellite data having much higher error range than land based temperature data.
It would be really interesting if someone would find each of these guys who publish government-funded papers without publishing their data – and ask for our money back. Or else.
“You see, while you seem to have written a paper of some sort, your evidence is… not here. So you can either supply the actual, raw, unedited data you started with, or talk to the IRS. Today.”
It is the journals responsibility to ENFORCE its data standards.
PLOS for example demands that the data be archived, and they Check
At least they did in our case
But then Skeptics also slam Open publications
A reputable scientist, in my book, would not need to be FORCED to do what is necessary to enable his reviewers to do the things which peer reviewing of scientific papers is supposed to be about – checking that the methods used in the paper is not completely wrong, and that the experiment performed is repeatable by independent researchers.
Independently confirmed by Berkeley Earth:
https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise
I think you are going to enjoy retirement. I know I enjoy how exciting your retirement is :)
Well, well.
Here we go again…
Pingback: NOAA Whistleblower reveals ‘Pausebuster’ scandal | Tallbloke's Talkshop
Heavy reading. Government bureaucracy at its finest. At least they weren’t able to gum up the works enough to stop Karl’s result from being published, because that was and remains an important note on biases, and nothing said here goes against the basic idea.
Wow,
Yup none of dem dere sciency calls for quality, or credibility, could stop the advocacy.
It was quality. The results stood up and were published in a timely way. The system worked in favor of science.
“results stood up”!!! Yup – just like Mann’s results stood up. Right up to the time his “hide the decline”, misuse of stats., using known inappropriate data sets and several other things were realized.
Jim D – I am a little confused, please help me. Can Karl’s results be reproduced, or do they belong in the Journal of Irreproducible Results?
Zeke Hausfather reproduced them independently on this blog a short time ago.
Jim D, thanks.
Jim D | February 5, 2017 at 8:28 pm |
“Zeke Hausfather reproduced them independently…”
Could also be phrased as “Hausfather treated the same data with similar results because both teams made the same errors at the same places in the process”
The starting material is from the same sources. There might be a need to ask questions if the results WERE different..
“and nothing said here goes against the basic idea.”
Aside from the whole thing being totally corrupt, of course.
But hey, that doesn’t matter to you does it, Jimbo?
After all, in JimboWorld the end justifies the means, right?
It looks like an attempt to red tape a piece of work to death. It failed.
It looks like somebody tried to get the “researchers” to justify bad findings before the organisation’s reputation was trashed. And don’t think for a moment that NOAA’s reputation hasn’t been trashed. Skeptics know what passes for science at NOAA and despise them for it. True Believers know what NOAA got away with, but just think it was justified because it supports the narrative. But they don’t trust them with real science.
Huh. Could have sworn this post argues the paper should not have been published. Did we read the same post?
He has nothing against the actual results that I saw. His complaint is only that it was too early for him.
Guess you missed this part:
“So, in every aspect of the preparation and release of the datasets leading into K15, we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation.”
That looks like an opinion that has not been substantiated with any results.
Wow, you are in denial Jim. You’re right, he has nothing because the data wasn’t archived and has gone missing. How convenient!
He can get his own data, right? Buoy data can’t be that hard to find. What stops him from doing his own work? These complaints never come up with an alternative result, and unless they do, the whole complaint is on no foundation.
So Karl had his thumb on Steve Mosher and Richard Muller?
Preposterous.
There is no there here.
Um, he’s retired.
JCH, “So Karl had his thumb on Steve Mosher and Richard Muller?
Preposterous.”
Ideally, products would be completely independent and use consistent methods. When you peek at your neighbor’s paper and decide your method has to be “improved” because it isn’t keeping up with the Jones, it isn’t really independent. When you decide the time to “improve” happens when you can make the biggest splash, your ethics become questioned.
Kind of a rock and a hard spot though, don’t change and your original “perfect” way of doing things sucks, do change and it is obvious your original way of doing things sucked. It is a lot better to pick the right course right out of the gate.
A lone voice in the wilderness? Sad.
“That looks like an opinion that has not been substantiated with any results.”
https://geosciencebigpicture.files.wordpress.com/2017/02/pausebuster-dataset.jpg
Why would a NOAA person be pointing to Met Office data? Or is he?
“That looks like an opinion that has not been substantiated with any results.”
I have described below what a thoroughly dishonest graph that is (from David Rose of course). The difference he shows is almost entirely due to the fact that HADCRUT is on 1961-90 base; NOAA is 1901-2000. If you put them on the same 1981-2010 base, there is hardly any difference.
“You’re right, he has nothing because the data wasn’t archived and has gone missing. How convenient!”
Again, just nonsense. Bates even linked to the archive. It is here.
“So Karl had his thumb on Steve Mosher and Richard Muller?”
Ya that’s a funny one.
Long ago I FOIA’d him over him shutting off access to metadata.
access was re instated. I’m pretty sure he didnt take too kindly to that.
It is true that the Karl paper is an important note on bias, but not in the way I think you mean. It is a prime example of how bias can destroy proper application of the scientific method, create a “result” that is worse than meaningless because it was established a priori, which then creates positive feedback for the corruption of science by others investigators as well. This particle example of work carries the added infamy of doing this in both the technical and political realms, infecting both scientists and policymakers.
In other words, it is a perfect microcosm of its field.
You can try a scientific argument against the ship-buoy bias, and people may listen, but this kind of argument has no substance.
Jim D.
One thing that has repeatedly Shocked me is the way the temperature record DOESNT CHANGE despite the changes in processes, methods, and data. I cannot count the number of times I have left data “on the floor” and had the answer come out the same.
That said. Now that the commerce department is run by Trump it should put an end to the claims of fraud. The CDR work is pretty much check box work.. you can tell how vital it is to skeptics by their insistent demands that Christy and Spencer update their CDR.
Or you can see how important it is by comparing the RSS CDR ( which is excellent but a bit out of date ) to the UHA CDR which is a out of date mess.
See how many skeptics
A) have ever READ the CDRs or know here to find them
B) have decided that RSS is preferable because of its CDR.
“…One thing that has repeatedly Shocked me is the way the temperature record DOESNT CHANGE despite the changes in processes, methods, and data…”
You’re joking, right? Compare the “null” process (i.e., raw data) to the processed data and it “DOESNT CHANGE?” Really? Then why bother homogenizing, processing data, accounting for station moves, etc, in the first place? Why is BEST still making updates?
“You’re joking, right? Compare the “null” process (i.e., raw data) to the processed data and it “DOESNT CHANGE?” Really? Then why bother homogenizing, processing data, accounting for station moves, etc, in the first place? Why is BEST still making updates?”
1. Yes it doesnt change. The answers will be different here and there
But the relevant scientific information remains the same. The surface
of the planet is warming. The LIA was real. To be Sure if you use
100% RAW DATA the rate of warming will be higher. ADjustments
COOL the entire surface record, they do not warm it.
2. Why Bother?
The first reason is that Skeptics had a hypothesis about adjustments.
They hypothesized that The scientists had cheated on the land record. So they PAID US to find the cheating and prove that the adjustments were wrong. We found the opposite.
The next reason is that we want the most accurate record possible.
While the raw data is actually warmer than the adjusted data ( when you look at SST and SAT ) it’s important to be as accurate as you can be.
3. Why are we still making updates? Simple, because you can never be correct you can only be less wrong. And every day I ask.. how can I be less wrong. Now, Nothing in the Science changes with these minor adjustments.. The world is still warming.
Steven Mosher.
Well said.
I can’t find the file at the moment, but it has been published on a reliable skeptic site recently. It shows the gigantic changes made at NOAA and GISS to the climate history. These are scans of the original published works and they show the extremes that they have gone to to wipe out the warm ’30s.
Steven Mosher:
You assert,
“One thing that has repeatedly Shocked me is the way the temperature record DOESNT CHANGE despite the changes in processes, methods, and data. I cannot count the number of times I have left data “on the floor” and had the answer come out the same.”
The data changes almost every month with e.g. this result
http://jonova.s3.amazonaws.com/graphs/giss/hansen-giss-1940-1980.gif
Indeed, a paper which a group of us attempted to publish on the matter was prevented from publication by the frequency of the changes as is explained in detail in a submission to a UK Parliamentary Select Committee and calls global temperature MGT; see
http://www.publications.parliament.uk/pa/cm200910/cmselect/cmsctech/memo/climatedata/uc0102.htm
.
The submission includes this,
“7.
However, the compilers of the MGT data sets frequently alter their published data of past MGT (sometimes they have altered the data in each of several successive months). This is despite the fact that there is no obvious and/or published reason for changing a datum of MGT for years that were decades ago: the temperature measurements were obtained in those years so the change can only be an effect of alterating the method(s) of calculating MGT from the measurments. But the MGT data sets often change. The MGT data always changed between submission of the paper and completion of the peer review process. Thus, the frequent changes to MGT data sets prevented publication of the paper.
8.
Whatever you call this method of preventing publication of a paper, you cannot call it science.
But this method prevented publication of information that proved the estimates of MGT and AGW are wrong and the amount by which they are wrong cannot be known.
(a) I can prove that we submitted the paper for publication.
(b) I can prove that Nature rejected it for a silly reason; viz.
“We publish original data and do not publish comparisons of data sets”
(c) I can prove that whenever we submitted the paper to a journal one or more of the Jones et al., GISS and GHCN data sets changed so either
the paper was rejected because it assessed incorrect data
or
we had to withdraw the paper to correct the data it assessed.
But I cannot prove who or what caused this.”
PLEASE NOTE: This explanation is quoted from Hansard and if untrue it would be a perjury that would have put me in jail.
Also, one of the emails from me that was leaked by the Climategate whistleblower objects to this method that was used for blocking the paper.
Richard
“The data changes almost every month with e.g. this result”
That is not the result of data changing every month. It is the result of totally misrepresenting the data, using plots of different things.
The first plot is also in a paper of Hansen in 1981. It isn’t GISS data (it’s NCAR, Jenne), and it is a land stations plot. Of the data, Hansen said:
https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/hans81.png
Very little SH data.
The second is a plot of an early version of GISS met-stations only. This aims to give a global temperature by re-weighting met-stations to represent oceans as well. It was becoming possible through the larger dataset available. This lives on as GISS Ts.
The third set is different again. It is a land/ocean set, which combines land stations and SST. The three are simply not the same thing. It is nonsense to say that they are the symptom of monthly changes.
Very little SH data.
And little or no in situ, or satellite coverage of sst in the climatological significant southern ocean, till around 2002 (S. Kravtsov et al 2011)
Lots of imaginary and bogus data.
“Lots of imaginary and bogus data.”
He actually did remarkably well with what was available. But yes, there is vastly more now. That is why it is just stupid to attribute the difference between a 1980 plot and a modern one to some kind of malfeasance.
Nick Stokes:
Thankyou for confirming my point. As you say, the compilers of global temperature time series can and do change their data depending on what they want to present.
As I said, these changes are possible because there is no agreed definition of global temperature.
All global temperature data will continue to be bunkum until
(a) there is an agreed definition of global temperature
and
(b) there is some possibility of an independent calibration standard for global temperature.
Until then all global and hemispheric temperature data remains less scientifically valid than phrenology. Paymasters say what they want and the compilers of global and hemispheric temperature data can and do provide it.
Richard
Richard,
“As you say, the compilers of global temperature time series can and do change their data depending on what they want to present.”
I didn’t say that. Here they are just presenting what is known at the time. Hansen in 1980 wasn’t compiling a time series; he was using someone else’s data in writing a paper about GHE. And in 1987 he was initiating one, but did not use SST because a suitable series wasn’t available. That came later.
But yes, scientists do look at different datasets for different reasons. land only, NH, SH etc. What is stupid or worse is when people pick out these graphs JoNova-style without looking at whether they represent the same thing and say – but you keep changing the data.
That is why it is just stupid to attribute the difference between a 1980 plot and a modern one to some kind of malfeasance.
Well something is not kosher at present in the SH with adequate station data.
https://data.giss.nasa.gov/tmp/gistemp/NMAPS/tmp_GHCN_GISS_ERSSTv4_1200km_Anom12_2016_2016_1951_1980_100__180_90_0__2__POL/amaps.png
Nick Stokes:
In your attempt to ‘hide the pea’ you are denying your own words.
The three graphs each purport to be global temperature and you correctly said they are “plots of different things”.
YES! THEY ARE PLOTS OF DIFFERENT THINGS. THAT IS THE POINT.
In response to your saying they are “plots of different things” I wrote,
“Thankyou for confirming my point. As you say, the compilers of global temperature time series can and do change their data depending on what they want to present.
As I said, these changes are possible because there is no agreed definition of global temperature.”
Your response to that is to ‘move the pot containing the pea’ by claiming,
“I didn’t say that. Here they are just presenting what is known at the time”.
NO!
The measurement data from earlier times were and are a record.
“They” used different compilation methods to generate the different graphs of global temperature from the same available record of measurement data. A unique definition of global temperature would define how measurement data would be compiled to generate each value of global temperature in the time series.
IN REAL SCIENCE A PARAMETER PROVIDES AN INDICATION OF REALITY. IT DOES NOT SUGGEST WHATEVER ITS PRODUCERS WANT IT TO SUGGEST AT ANY GIVEN TIME.
THE GRAPHS DIFFER BECAUSE THEY REPRESENT THE DIFFERENT SUGGESTIONS THEIR COMPILERS WANTED THE TIME SERIES OF GLOBAL TEMPERATURE TO SUGGEST AT THE TIMES THEY WERE PRODUCED.
Indeed, your original post admitted this saying,
“The third set is different again. It is a land/ocean set, which combines land stations and SST. The three are simply not the same thing.”
YES!
THEY ARE NOT THE SAME THING BUT THEY EACH PURPORT TO BE A TIME SERIES OF GLOBAL TEMPERATURE!.
Indeed, the selection of which data to use from which surface reqions is the masking issue that I addressed in the link I provided and discussed in the post you replied (for your convenience I here repeat that link http://www.publications.parliament.uk/pa/cm200910/cmselect/cmsctech/memo/climatedata/uc0102.htm )
Appendix A of that item is one of the emails from me that were leaked by the Climategate whistleblower and it includes this,
“.. as my response states, Myles’ comments do not alter the fact that the masked data and the unmasked data contain demonstrated false trends. And the masking may introduce other spurious trends. So, the conducted attribution study is pointless because it is GIGO. Ad hominem insults don’t change that.”
And your attempt to claim you said other than you wrote also doesn’t change it.
To ensure your attempt has not hidden it, I again state the important issue from which you have attempted to deflect attention,
“As I said, these changes are possible because there is no agreed definition of global temperature.
All global temperature data will continue to be bunkum until
(a) there is an agreed definition of global temperature
and
(b) there is some possibility of an independent calibration standard for global temperature.
Until then all global and hemispheric temperature data remains less scientifically valid than phrenology. Paymasters say what they want and the compilers of global and hemispheric temperature data can and do provide it.”
Richard
Lets not lose track of the ball here. One takeaway from John Bates’ post is that government science agencies – or at least NOAA – do take integrity of the data, archiving and access seriously. Do not let the actions or behavior of some color your conclusions.
Another is that Dr Bates shines a light on the real importance of the Karl paper. And it is not its refutation of the pause or any other aspect of climate science. As Mosher has pointed out in the past, if you accept Karl’s conclusions, you also have to accept that the rate of warming was lower than the “consensus” was claiming. My takeaway from Karl was a shot in the foot from trying to be fastest on the draw. (note to Western enthusiasts – always choose a rifle, then a shotgun. A pistol is your last choice. Also, if you are in a pistol fight, be prepared to get shot. Putting your first round on target is by far the most important thing in a civilian fire fight, not who draws and fires first.)
The importance of the Karl paper will be as a text book example of what happens when you already have a specific result for your research. It will be a shame if an otherwise good scientist’s claim to fame is the adding the verb Karlization to the scientific vocabulary. But it’s a bed he made.
The paper’s timing was highly suspect. After 30 years of watching the USA government and media distort facts and create bogus information I tend to distrust all information they put out which may be intended to create support for political objectives. This means the Karlized data sets don’t convince me at all.
I realize mine is a fairly simple minded and “unscientific” approach. But unfortunately global warming is now a political hot potato. It’s going to be, whether we like it or not, a battlefield in the 21st century Obamite-Trumpite Disinformation Civil War.
the integrity of acquisition archiving and access is taken seriously but there are a lot of logistics that have to align that don’t …and honestly NOAA doesn’t take its fisheries ship techs seriously and its mostly due to the inconsistency of the NOAA corps officers who are only managing the civilian techs for a ‘management grade’ on their evals without having any background in surveying themselves and aren’t in the position long enough to be accountable for their own naive mistakes in judgment
Actually Jim it is an extremely clear and well written piece, despite having to utilize the technical nomenclature of the various data sets.
Do you know what goes into each of the datasets described here, and which you would trust more than the other? Without that background it is just a list names with version numbers, and few could be bothered to go and review that background. It really is in the weeds there. Karl’s basic idea supersedes all this because it is understood only in terms of ship and buoy biases. Plain and simple.
What’s wrong Jim? Once again having to admit you haven’t a clue about what you are talking about and having to change the subject?
Why do I need to know every detail about the data sets themselves? As for which would I trust more, based on what I do know? I believe the answer to that depends on what one wants to use the data for. But just for the sake of argument, I would say the Argo data is probably the better set. Just hasn’t existed for that long. Hence the use of historical ship collected data. How about you explaining the logic behind taking what is considered to be the more reliable data set and adjusting it using the less reliable set?
The odd thing is that Bates thinks the ERSSTv4 was handled properly by Karl et al., and that is where his main result was (ships versus buoys). His complaints are confined to land data, and some quibble about machine-readable where no one understands what he is talking about.
a CTD has dual redundant temp sensers
calibrated annually, rinsed after each opporeration, and its comparative plot readings are monitored closely during the cast
An Argo is deployed with a drifting life expectancy of five years …sensors drift over time when not calibrated regularly
Buoys are left to drift and often cant be serviced as timely as one would like
A thermosalinogtraph is in a large scale piping system its temp sensors are spaced far apart and is often not monitored well on various different boats as it is turned on and left to run in the background with often only a digital read out instead of a relevant comparative plot readily displayed
Interesting, let me know if we need a new thread on this topic, and let me know if you would like to contribute a guest post. Thanks for your contributions here.
Too funny. Defend the pause to the last man. Deadenders to the dead end.
Do you fancy becoming “NOAAGate whistleblower” #n SIB?
Jim D’s basic point is this : it doesn’t matter that ethics and integrity were ditched in order to facilitate a preferred political announcement, because later examination upheld the findings.
Except when you note that the later findings were themselves heavily criticized for nothing but being political crap.
He published science which he knew was right and didn’t let delaying tactics deter him. If he had been wrong or premature, he would have deserved criticism, and later publications would have corrected it. Science is never complete and papers can only give a current status. You publish what you have at a given time, and can’t answer all the questions, or wait till you have all the answers. Papers always end with remaining questions and are never tied up with a bow.
Punksta, the very idea that fleet data, which includes bucket data, without the benifit of a full and comprehensive survey, became a trusted dataset is overwhelming my ability to hold down lunch. Karl et al 2015 is a dogs chew toy. There is no set of corrections which makes fleet data usable without intimate knowledge of each and every vessel and that is just the beginning
This statement by Jim D:
“He published science which he knew was right ”
Explains a lot.
Later studies backed him up, so he was right to publish. The main result was for the oceans. Bates’ quibble was for the land, so the main result was never in dispute.
Owen mentions intimate knowledge of the vessels …I was the Tech on two of the vessels 2006-2015… i was the one collecting equatorial SST via SBE21, SBE3,SBE38, SBE45 Thermosalinographs and an SBE9plus water sampler carousel CTD on the TAO el niño array runs as well as elsewhere and I deployed the most Argo floats of the fleet, and I knew the NDBC/SAIC buoy techs…
Read lengthy expose by Bob Tisdale on Karl’s malfeasance/misfeasance:
https://bobtisdale.wordpress.com/category/noaa-ersst-v4/
Oh, “But he isn’t one of us, so he can’t possibly know anything.”
You posers.
“Read lengthy expose by Bob Tisdale on Karl’s malfeasance/misfeasance:”
So what did Bob actually expose? All I see are endless graphs showing that NOAA has higher trend that HADSST. Yes, NOAA adjusted for the clear bias in matching buoy and ship data, and HADSST didn’t, at least not yet. Doesn’t mean NOAA is wrong, let alone malfeasant.
I see Bob Tisdale using data that bates claims was not available.
Bates is misrepresenting the truth
Mosh
Why would bates want to misrepresent the truth?
Tonyb
> Why would bates want to misrepresent the truth?
Moshpit’s claim doesn’t presuppose any want, TonyB.
Reblogged this on Climate Collections.
Perhaps it is worth mentioning that the standard engine room inlet temperature measuring instrument will usually be calibrated to EN 13190 class 2 standard, ie ±2 degrees centigrade when installed, and is highly unlikely to be checked during the lifetime of the vessel.
de-de.wika.de/upload/DS_IN0007_GB_1334.pdf
The logic of utilising readings from a device of such poor accuracy when Argo buoy instrument data accurate to 0.01 deg C is available is not clear.
Nor is it clear how readings from such gauges can be quoted in the Karl datasets to 0.001 deg C.
Mind you, I’m only an engineer, not a climate scientist.
Argo data agrees with K15
Last time here at Climate Etc Folks suggested we look at Argo and compare it to K15.
Guess what?
K15 is right.
Guess who suggested that we compare K15 to Argo, Satellites and Buoys?
Guess?
Skeptiics here at CLimate Etc
And what do you think they expected to find?
Well, What folks found was that K15 Adjustments are SUPPORTED and CONFIRMED by argo, satellites and buoys.
Mr. Mosher, ARGO was “adjusted” for engines and buckets, not the other way around [as one would do to provide a better look at reality].
Go Wander in another Weed Patch.
“Argo data agrees with K15”
So Argo data with an accuracy of ±0.01 deg C agrees with engine inlet temperature with a gauge accuracy (plus a collection of other variables) of 2 deg C, permitting temperature data to be quoted to three decimal places and you can’t see any issue with that…
Why am I not surprised…
I hope no-one ever puts you in charge of a real project, designing a supermarket trolley for example…
David. Argo wasn’t used in K15
Which set of Argo data are you referring to Mosher?
And as I tend to give your opinion a good deal weight on certain topics, can you explain the reasoning behind adjusting a data set considered to be more accurate using a less reliable set? If Argo is more accurate, why adjust it at all? Why didn’t Karl apply the same weighting of Argo without adjusting it first? His decision might be justifiable in the field of statistical analysis, but it sure doesn’t sound to be so in the field of common sense.
cat weasel …it’s not engine water intake temp …it is ‘uncontaminated salt water pump system intake’ on the bow which takes a water temperature sensor in the bow thruster room and compares it to another temp sensor farther up the line usually in a wet lab
Here is the example of an SBE38 and its specs
http://www.seabird.com/sbe38-thermometer
Please be aware that I am describing not research vessel measuring technique, but standard merchant marine practice, as is described here:
.
http://www.nature.com/climate/2008/0809/full/453601a.html
Those merchant vessel measurements – both bucket and engine room inlet water temperature – make up the vast majority of the sea surface temperature data over several decades.
Ah!!! Ok, thank you…cool article
“when Argo buoy instrument data accurate to 0.01 deg C is available is not clear.”
Argo buoys come to the surface once every ten days. They are accurate, but long gaps.
The 2011 paper on which Karl et al is based matches ships with buoys that are upto 50 km apart. LOL.
“that are upto 50 km apart. LOL”
You seem to think that is too far. Have you an estimate of the appropriate distance? Could you say why?
There is a trade-off in error of the bias estimate. Close encounters are better correlated, but there are fewer of them. A larger sample diminishes variance of the mean. They have done an extensive study of spatial correlation in part 1 of their study. They have a quantitative basis for their choice. You are arm-waving.
Ok, now I suspect you are here practicing your standup comic act Nick. “once every ten days … there are gaps” That’s funny.
Aren’t there rather huge gaps in the ship intake data? Like almost everything outside of shipping lanes?
“Argo buoys come to the surface once every ten days. They are accurate, but long gaps.”
Now that’s funny Nick. Are there not some rather large gaps in the ship intake data? Like everywhere outside of shipping lanes?
Shub and Tim
Yes there are long gaps in ships data when seastates cause the intake to suck air and we shut it off to save the pumps..
or when an part breaks underway that can’t be fixed for a while
or when transiting areas where we don’t want to clog the intake with oil or algae etc
As for comparing data between ships and buoys 50km apart, we collected ships data and buoy data when servicing the bouy..,so there are plenty close aboard measurements of ships data, buoy data, ctd data, and argo data all from the same operational stop, so if he was using data as far as 30NM …half a degree…that makes sense too as we leap frogged buoys if they didn’t need service…but we still would have steamed by …so there must have been extenuating circumstances
Punksta, the very idea that fleet data, which includes bucket data, without the benifit of a full and comprehensive survey, became a trusted dataset is overwhelming my ability to hold down lunch. Karl et al 2015 is a dogs chew toy. There is no set of corrections which makes fleet data usable without intimate knowledge of each and every vessel and that is just the beginning
I have intimate knowledge of the vessels, the buoys, and the Argos
Catweazle666, thank you for your comment. You illustrate nicely, why crap fleet data cannot be used. You have provided a single example, a full fleet survey would unearth a score more. And we havent touched on procedure yet. Resolution in this case cannot be guessed at. Nick Stokes, and Mosher, both of you are invited to see the forest through the trees. Karl et al 2015 cannot be defended, unless be a lunatic. I point to Nick and Mosher both because i respect you both.
Karl 2015 was not the paper that justified use of fleet data. That has been used for a very long time, but is being replaced by better buoy data. K15 is the paper that says how the buoy data can be properly used with ship data. Buoy data is greatly upweighted because of its lower variance.
Nick,
Not true about lower variance when the dominant vasriancebis from natural factors like stratification, mixing,diurnal cycles samples random times. In such cases it matters little which of 2 competing instrument systems is used.
Geoff,
The cause of variance doesn’t matter – they just weight according to what is observed. It’s a math process – you work out the uncertainty of combinations, then work out the weighting that minimises uncertainty.
“Buoy data is greatly upweighted because of its lower variance.”
Yep, after it has been adjusted using data of higher variance. You left that part out Nick.
*giggle* don’t get me started on procedure and other shananogans
We used oceanographic Seabird Electronics SBE21 and SBE45 Thermosalinographs with associated intake sensors (SBE 3 and SBE38 respectively)
They were calibrated annually and as needed and generally on a port and starboard rotation to be swapped during winter inport/dry dock periods
The limitations were such things as the flow gages…older set ups had manual valve flow gages with no read outs, and visiting scientists who were knob twiddlers were notorious for adjusting water flow without permission …or syphoning water without permission
The study drew criticism from other climate scientists, who disagreed with K15’s conclusion about the ‘hiatus.’ (Making sense of the early-2000s warming slowdown). The paper also drew the attention of the Chairman of the House Science Committee, Representative Lamar Smith, who questioned the timing of the report, which was issued just prior to the Obama Administration’s Clean Power Plan submission to the Paris Climate Conference in 2015.
There
So let’s make some guesses. These are the scientists who wrote the paper that disagreed with K15’s conclusions about the hiatus:
John C. Fyfe, Gerald A. Meehl, Matthew H. England, Michael E. Mann, Benjamin D. Santer, Gregory M. Flato, Ed Hawkins, Nathan P. Gillett, Shang-Ping Xie, Yu Kosaka & Neil C. Swart
Let’s guess what they would have to say about the above CargoCult Etc. hatchet job?
My guess is they will condemn this article in strongest possible terms, and that they will fully stand by their real assessment of Karl15:
Recent research that has identified and corrected errors and in homogeneities in the surface air temperature record (4) is of high scientific value.
4. Karl, T. R. et al., Possible artifacts of data biases in the recent global surface warming hiatus. Science (2015).
.
As for why the WaPo rejected a shorter version of this article, they possibly contacted the K15 co-authors and got a very different story. Just a hunch.
John – If you’re around, I’d be most interested to hear your opinion on the methodology behind NOAA’s recent assertion that the “Arctic is Warming Twice as Fast as the Global Average”. See:
http://GreatWhiteCon.info/2016/12/arctic-sea-ice-news-from-agu/#ReportCard
I’d also be interested to hear your views on David Rose’s reporting of “ClimateGate 2”!
Clutching at straws Hunt?
I think I’m safe in assuming that your name isn’t John Bates?
Am I right or am I right?
Although this doesn’t respond to your question about methodology this is also what the UAH satellite data says.
For the arctic, 60 to 85 latitudes, the trend is +0.24 deg C over the last 39 years.
For the antarctic, 60 to 85 latitudes, the trend is 0.00 deg C over the last 39 years.
For the globe as a whole and for the tropics, 20 to 20 latitudes, the trend is 0.12 deg C over the last 39 years.
All those trends are per decade. Ie. 0.12 deg C per decade.
==> There are global surface temperature datasets, such as Berkeley Earth and HadCRUT that are relatively independent of the NOAA data sets, that agree qualitatively with the new NOAA data set. ==>
Kudos to you, Judith, for mentioning this important caveat regarding the implications of the main post.
it is unfortunate to note that Rose’s article about how world leaders were “duped” fails to include that comment of yours, or indeed anything about BEST and charts “flawed” NOAA and HadCRUT data but didn’t include BEST. Perhaps you might comment to Rose about his oversight.
Also, while I note that the author (in the Rose article) says: “‘I want to address the systemic problems. I don’t care whether modifications to the datasets make temperatures go up or down.” , it is unfortunate, IMO, that he did not provide a similar reference point as you provided, or better yet provide his own analysis to show a materially different results than Karl’s analysis – .to back his accusations of scientific fraud (“putting [a] thumb on the scale”).
I’ll raise those matters with Mr. Rose’s managing editor on your behalf Joshua, if that would be helpful?
Before The Pause was erased by Karl15:
In 2011 an article at Greenwire was based on a dozen climate scientists being asked the question:
“Why, despite steadily accumulating greenhouse gases, did the rise of the planet’s temperature stall for the past decade?”
There was no doubt that a slowdown in global warming had occurred. Kevin Trenberth:
“The hiatus [in warming] was not unexpected. Variability in the climate can suppress rising temperatures temporarily, though before this decade scientists were uncertain how long such pauses could last. In any case, one decade is not long enough to say anything about human effects on climate; as one forthcoming paper lays out, 17 years is required.”
https://pielkeclimatesci.wordpress.com/2011/10/27/candid-comments-from-global-warming-climate-scientists/
In August, 2014, the New York Times discussed the attempts to understand The Pause. Andy Revkin:
“There’s been a burst of worthy research aimed at figuring out what causes the stutter-steps in the process [global warming] — including the current hiatus/pause/plateau that has generated so much discussion. The oceans are high on the long list of contributors, given their capacity to absorb heat. The recent studies have pointed variously to process in the Pacific and Atlantic and Southern oceans……”
https://dotearth.blogs.nytimes.com/2014/08/26/a-closer-look-at-turbulent-oceans-and-greenhouse-heating/?_r=0
Also in August, 2014 Judith Curry discussed papers which tried to explain The Pause.
“Atlantic vs Pacific vs AGW”
https://judithcurry.com/2014/08/28/atlantic-vs-pacific-vs-agw/
I asked the New York Times to do a story about the govt only vessel solely dedicated to climate studies being pulled off line in 2012 on short notice due to the across-the-board budget cuts…they were interested in the story but asked “Is there a conspiracy?” When I said “No, just unfortunate budget cuts” he was no longer interested in the story
Pingback: Climate Science & Data Management - From The Prow
the alarmists are on their heels with this… one more good push.
This was supposed to be big. It’s not quite nothing, but close.
Not quite nothing? This is not an expose’ on the results but it’s damning of the players and the game.
“Thumbs on scale”
Not documented and utilized a defective data set. “The incident report had found that there were unidentified coding errors in the GHCN-M processing that caused unpredictable results and different results every time code was run.”
Did not follow the prescribed guidelines of the scientific organization nor the publishers. According the Daily Mail article potentially subject to retraction.
Not reproducible.
And you don’t see anything wrong with this?
There are two sides to a story… look at the way the Fyfe paper is being hyped.
Peterson says he’s not aware of any issues raised in the months just prior to the publication of the Science study. But in 2013 and early 2014—well before the disputed study was submitted to Science—Peterson says there was tension between agency scientists and data managers. The scientists wanted to publish a paper based on a then-new, more comprehensive database of land temperatures from the ITSI. Others in the agency pushed for a delay out of concerns the new ITSI data hadn’t fully met NOAA protocols for releasing such databases to the public. The dispute led to a 6-month delay in the publication of that earlier study in the Geoscience Data Journal, says Peterson. The ITSI data was later used in the Science study. …
JCH,
Presume you’re referencing this: http://www.sciencemag.org/news/2015/11/researchers-dispute-lawmaker-s-allegation-noaa-rushed-climate-study
Now, under new management, I’d suspect that the communication will become available. Would it be out of order to expect that should that communication indicate there were time ‘pressures’ and/or other ‘less than scientific reasons’ for a rush to publication that the reputations of the scientists and certainly NOAA won’t be tarnished. And presuming that to be the case (maybe incorrectly) how will that be beneficial?
Maybe they already have all of them.
Maybe? Based on?
Again, this expose’ in no way suggests the Karl work is incorrect but it does appropriately state that the methodology and associated communication should see greater light of day.
In this case, maybe (since we’re tossing that word around) Mr. Lamar Smith is doing the peoples work. If indeed those communication are in hand they should be viewable. It’s our work product.
I’m still reading through my 100s of thousands of IRS documents that some maroon congressman demanded.
From Rose’s article:
==> It’s not the first time we’ve exposed dodgy climate data, which is why we’ve dubbed it: Climate Gate 2 ==>
Can someone remind me whether this is the 2nd “Climategate 2” or the 3rd one?
in terms of significance, all climategate material is in the gate archive between billygate and coingate.
You really don’t think this is the end of it.
Of sour grapes? No.
Trump is going to be over the moon at this.
Talk about playing directly into his hands!
Just another wow. Atomic wow.
John- Thank you for doing this. Your experience with inappropriate behavior by Tom Karl, Tom Peterson and Peter Thorne at NCDC is consistent with my experiences with the CCSP 1.1 Report. If you have not read these, you might find them informative
1. Pielke Sr., Roger A., 2005: Public Comment on CCSP Report “Temperature Trends in the Lower Atmosphere: Steps for Understanding and Reconciling Differences”. 88 pp including appendices. http://pielkeclimatesci.files.wordpress.com/2009/09/nr-143.pdf
Here is just one excerpt from the Executive Summary of my statement
“The process that produced the report was highly political, with the Editor taking the lead in suppressing my perspectives, most egregiously demonstrated by the last-minute substitution of a new Chapter 6 for the one I had carefully led preparation of and on which I was close to reaching a final consensus. Anyone interested in the production of comprehensive assessments of climate science should be troubled by the process which I
document below in great detail that led to the replacement of the Chapter that I was serving as Convening Lead Author. ”
2. Further Documentation Of Inappropriate Behavoir By A Subset Of Members Of The CCSP 1.1 Committee And The NRC Review Committee
https://pielkeclimatesci.wordpress.com/2011/11/30/further-documentation-of-inappropriate-behavoir-of-members-of-the-ccsp-1-1-committee-and-the-nrc-review-committee/
Here is just one extract from the Climategate e-mails
“Phil Jones wrote:
BEN WAS REALLY PISSED OFF WITH ROGER — AS WAS TOM KARL I GUESS (NOT YET TALKED TO HIM).ALL OF HIS POINTS CAN BE SHOT DOWN, BUT IT IS A PAIN NONE THE LESS. APPARENLTY JUDY CURRY EXPOSED HER INFERIORITY COMPLEX (ANS HER INFERIORITY).”
Your new work has further exposed a very manipulated effort for that community.
Best Regards
Roger Sr.
And so the evidence builds, From outside and now inside, And also now Clitzbe supposedly has many other solicited climate whistleblowers.
Hello!
A quick question, requiring an answer only from your five senses.
Has it been a warm winter where you are? or not? Was last summer warmer or cooler than normal? I know you are trying to measure something far more complex, but I’d be interested to hear your reactions.
a bit warmer in the summer, definitely colder in the winter…and your point being?
An additional important issue is NOAA’s compliance or lack of compliance with federal peer review procedures for influential scientific information.
Under the OMB Peer Review Guidelines together with NOAA policy implementing the OMB Guidelines , scpub201506 was clearly “influential scientific information” and Karl et al 2015 was a “highly influential” scientific assessment. OMB Peer Review Guidelines established formal procedures for peer review of “influential scientific information” (Part II) and “highly influential scientific assessments” (Part III). NOAA policy states explicitly that “Peer review of these [influential and highly influential] information products will be conducted in accordance with that Bulletin.”
Under these policies, NOAA was obliged, inter alia, to establish a peer review record accessible to the public. NOAA resisted Lamar Smith’s requests on the questionable grounds that scientific correspondence was privileged, but this is not the case for the peer review record of influential scientific information.
+100
I questioned another co-author about why they choose to use a 90% confidence threshold for evaluating the statistical significance of surface temperature trends, instead of the standard for significance of 95% — he also expressed reluctance and did not defend the decision.
Reminds me of the 1993 EPA 2nd Hand Smoke study.
The 90% threshold bothered me the most. This could make a huge difference in interpretation and the wording of results. When dealing with measurements ostensibly accurate to 0.01 or even 0.001C this changes designation of whether or not any observed change is meaningful. If 95% is the standard for climate data it should be used. The only other departure allowed should be the actual level attained rather than a threshold level.
The AR5 best-estimate ERF trend over 1998–2011 is 0.23 ± 0.11 W m–2 per decade (90% uncertainty range), which is substantially lower than the trend over 1984–1998 (0.34 ± 0.10 W m–2 per decade; note that there was a strong volcanic eruption in 1982) and the trend over 1951–2011 (0.30 ± 0.10 W m–2 per decade; Box 9.2, Figure 1d–f; numbers based on Section 8.5.2, Figure 8.18; the end year 2011 is chosen because data availability is more limited than for GMST). The resulting forced-response GMST trend would approximately be 0.13 [0.06 to 0.31] °C per decade, 0.19 [0.10 to 0.40] °C per decade, and 0.17 [0.08 to 0.36] °C per decade for the periods 1998–2011, 1984–1998, and 1951–2011, respectively (the uncertainty ranges assume that the range of the conversion factor to GMST trend and the range of ERF trend itself are independent). The AR5 best-estimate ERF forcing trend difference between 1998–2011 and 1951–2011 thus might explain about one-half (0.04°C per decade) of the observed GMST trend difference between these periods (0.06 to 0.08°C per decade, depending on observational data set).
Yes. There are a bunch of us who would just prefer that folks report the observed level of confidence and drop the term “statistically significant”
A few things.
1. The findings of K15, which Bates somewhat misrepresents, hold up under independent examination. In particular it was suggested here at Climate etc, that folks should look at other datasets to conform or call into question K5 decisions. ( thanks Dave Springer for suggesting folks compare it to argo) The hypothesis here at Climate Etc was that IF folks looked at
other data (like argo) they would see that Karl had his thumb on the scales.
Well, folks did just that. They looked at satellites, Buoys and Argo.
What did they find? They found that Karl’s thumb was weightless.
That is, they found that the independent data sets CONFIRMED the
adjustments. This doesnt make them perfect. This doesnt preclude another Look at the data. What it shows is that the Supposition that Karl’s adjustments would be shown to be inncorrect was Wrong. “falsified” if you
like that term. Busted. The Karl adjustments to SST are confirmed by looking at data folks have never looked at before. See zekes paper.
2. CDRs
I would like to thank Dr. Bates for Pushing for CDRs and a more formal process. I would hope that formal CDRs would get folks to stop their claims
of fraud. I would like everyone to hold themselves to CDR like processes.
It would be great if Christy and Spencer updated their CDR. It’s way way way out of date. Wouldnt it be great if Goddard, Watts, Willis, Ah hell everybody used a CDR like process?. It would great if folks didnt post charts unless they went through a formal process. But even with a CDR like process I am sure that people will still claim fraud. because they can. I know that people will still demand more more more. Not because they actually want to use the data, but just as a form of obstruction, diversion etc. Wouldnt it be great if Judith Required a CDR and up-to-date code repository for every post at Climate Etc.? Do you think she will?
in any case. when folks post their findings here on climate Etc.. you can expect me to ask
1. Where is your SVN or Git
2. Where is your data posted.
3. Where is your CDR or functional equivalent
I expect every intellectually honest person to agree with me and demand the same everywhere. not gunna happen.
Funny story. Long ago I ran a program to develop a UTD ( unit training device) basically a flight sim for training fighter pilots. It was great. We ended up winning a huge contract teamed with Hughes Training to build a UTD for the F16. Our code was all research code, but hey, bringing it up to snuff would be no problem. So I just paired every research programmer with a guy skilled in MilStd 2167A/T. After 3 days one of the research guys punched the standard’s guy in the face. Ouch.. thunderdome. This is just my way of saying that the move to a more production oriented approach won’t happen overnight and it wont be bloodless.
Steven,
As you’ve stated a sufficient number of times this is a blog.
“in any case. when folks post their findings here on climate Etc.. you can expect me to ask:
1. Where is your SVN or Git
2. Where is your data posted.
3. Where is your CDR or functional equivalent”
As a member of the scientific community surely you’re willing to ask the associated of the above referenced to those such as Karl. Based on your history, had you asked Karl and found that which is outlined in this expose’ might you have viewed the work through a different lens? Apparently his offering would not hold up to the standards you wish to apply here.
“As a member of the scientific community surely you’re willing to ask the associated of the above referenced to those such as Karl. Based on your history, had you asked Karl and found that which is outlined in this expose’ might you have viewed the work through a different lens? “Apparently his offering would not hold up to the standards you wish to apply here.”
I Viewed his data through the lens of a Skeptic. That is WHY
Folks on our team decided to Check his work by Looking
at OTHER DATA using OTHER METHODS.
For reference see Zeke’s comments here when K15 came out.
He withheld judgment.
People really need to think harder or read what I have written in the past more closely.
If you present an argument and refuse data or code, Then I am under no rational obligation to believe you OR find your mistake.
The absence of these things ( code, data, CDR) doesnt make you wrong.
It just means I’m under No obligation to CHECK your work or to believe it.
since we checked Karl’s work what do you conclude?
Mosher,
Show me a better chart of Phanerozoic Global Temperatures than this – i.e. one that has been passed and approved by the climatariate gate keepers:
https://html2-f.scribdassets.com/9mhexie60w4ho2f2/images/1-9fa3d55a6c.jpg
Explanation of how it was produced here: https://www.researchgate.net/publication/275277369_Some_Thoughts_on_Global_Climate_Change_The_Transition_for_Icehouse_to_Hothouse_Conditions
After 30 years of intense climate science, and ideology driven climate control policies which have done great damage to the global economy, why don’t we have a widely accepted chart of temperatures over the Phanerozoic Eon?
It is important because this chart, if correct, may mean the GCMs and IAMs are exaggerating the negative impacts and damages of GW.
Apparently, Mosher doesn’t have an answer for this question, so he dodges it. How surprising (/sarc)
Some reading material for Peter Lang. It doesn’t paint a pretty picture of a few degrees of warming. This is not something to be casual about.
http://globalwarming.berrens.nl/globalwarming.htm
Jim D,
I don’t see anything in those sentences about impact functions or damage functions, do you? It is just a pile of cherry picked factoids and opinion (that support your belief). It is the Alarmists’ equivalent of this:
A complete list of things caused by Global Warming
http://www.numberwatch.co.uk/warmlist.htm
You can see net damage, however. How you evaluate it in dollars depends on how you value lives in poorer regions. GDP alone doesn’t cut it because that values an African life at about 5% of an American one.
Your just babbling again Jim (Denier). There is no valid evidence that GW will do more harm than good. That is the point. If you disagree, provide valid evidence – not a pile of Alarmist’s assertions.
Read it again. It tells you what you keep saying you are not hearing, and maybe that is because your ears are closed to it.
No it does not. Read it again yourself. What is the damage function? What is the justification for it? What is the net-benefit or damage to the world by 2100 – E.g., 2015-2100 in 2010 US$ GDP?
Importantly, you need to provide a link to how the estimate was made, all the inputs, methods and assumptions.
You might like to get some background on the issue, since you clearly have not a clue about it:
Start with the comments follosing this: https://judithcurry.com/2016/11/25/week-in-review-science-and-policy-edition-3/#comment-826494
And this: https://judithcurry.com/2017/01/29/the-threat-of-climate-change/#comment-836115
Don’t respond until you’ve read them and understood the issue.
You are asking whether it is net benefit. It does not look like net benefit to me. As to the cost of livelihoods and damage, value it any way you like. There are many ways. That is your choice. Don’t wait for me to tell you. Do you want it to be 4 C warmer after reading that? I would value the difference between now and that pretty highly.
You haven’t read and understood what I referred you to, and haven’t addressed the question. You do not have a clue about the issue. Shut up until you do, if you can, which I doubt.
No, and I won’t. I’m done with telling you about the facts of net loss of climate change. Calculate it any way you want, and we can review what you have.
According to the FUND analysis (Tol, 2013, Figure 3 – free pre-publication version here: http://www.copenhagenconsensus.com/sites/default/files/climate_change.pdf ) GW is net beneficial out to around 4 C (ignoring energy consumption – for reasons explained in the linked comments your didn’t read or didn’t understand ).
All the advocated mitigation policies do far more harm than good this century:
https://anglejournal.com/site/assets/files/1727/lang_1.png
That would be a GDP based analysis, I guess. This has moral issues as I mentioned.
Morals are your beliefs. Irrelevant.
A point on economics. Note that one set of people gets the benefit and a while different set gets the costs, so unless you are going to tax the benefits to pay for the costs, you can’t count them.
You’ve made that silly statement a hundred time. It’s been pointed out to you that it applies to all policies. All policies have winners and losers. Rational policy delivers the maximum benefit. Virtually no policy has no losers.
Stop babbling Jim Denier. You haven’t a clue. Go back to watching the football.
You just haven’t thought it through. There is a cost. Who pays? A carbon tax or maybe you want a benefit tax. Where does the money come from in your world? Note that all carbon use is a cost-benefit calculation. A person only pays for fossil fuels to gain benefit. The hidden part is the cost downstream (see scenarios).
If you think you have a better way to quantify and present the global net-benefits of GHG mitigation policies than the globally accepted standard, please show the equivalent chart in units of measure you deem appropriate, provide links to the basis for it, method, inputs, assumptions, and all else needed to be able to understand it and reproduce it (as I di for the above chart: https://anglejournal.com/article/2015-11-why-carbon-pricing-will-not-succeed/ .
Actually I go with a consensus $40 per tonne, but that’s just because it seems to pay for a lot of the damage and adaptation, if not all. You need an identified revenue stream from carbon to at least offset these costs. How mathematically exact it should be is academic. Just the presence of revenue matters.
There is no consensus on any carbon price and no valid justification for it. if there was, the world would have one. As I said, you are just babbling. You don’t have a clue.
You haven’t identified how the future costs are paid for given that the damage falls to people who didn’t benefit. Isn’t that a flaw in your whole “net benefit” argument?
No. What counts is if there is a net benefit. That means the world is better off. Any policy has winners and losers. Even jailing a murder has winners an losers.
You think of the world as a single economy, not a set of independent ones. That’s very globalistic/idealistic of you. Hope it works like that but I think you need policies to get the money to the right people otherwise climate change justs sinks some nations.
You are correct that carbon pricing would create many more losers than winners and sink many nations – that’s why countries they will not participate unless they will incur net-benefits and why any attempts will not be sustained. This applies to any GHG mitigation polices that would incur net-losses to their economy.
Carbon pricing is a small addition to the cost to pay for its later damage and the costs of adaptation and mitigation too. Benefits take care of themselves. It is just balancing the books.
Now Peter, be reasonable. If Jim doesn’t post something until he understands the issue, he’d never get to write ever again. ^¿^
You tell him, schitz:-) I’m just watching the Superbowl ads.
JimD – Everyone has benefited from the burning of fossil fuels. Some have benefited more than others, but a benefit is a benefit. Get over it already.
Yes, people pay for the benefit part already. It’s the downstream cost part that needs adding.
Jim D,
I have noticed on a number of threads that you seem to have a fundamental misunderstanding about the net cost of decarbonization. You use phrases like “It is just balancing the books” which suggests you think this is a zero sum game. It is not. Decarbonization has a real cost to current GDP growth – as much as 2% on a net basis according to Stern. To use technical economic terms, that is freakin’ huge. The economic dislocation and societal impacts of this GDP loss (which will be mostly reflected in consumption) will be highly significant. If you want people to agree to sacrifice over $1 trillion today (and every year) to avoid potential costs in the future, you had better damn well have a reliable damage function. Peter Lang is 100% correct on this.
DavidT, yes, if you want to calculate the costs go ahead, but don’t offset them with benefits unless you also have a way to tax those benefits to pay for the costs, which you don’t. It is fundamental to understand that the people with the benefits are not the same people with the costs. And, no, it is not zero-sum. There is a loss either way. Mitigation and climate stabilization once and for all costs less than continuing costs for adaptation and damage for centuries.
Jim D,
I honestly don’t know what are are talking about. Can you explain further what you mean by “don’t offset them with benefits”? I think you also miss the point. It is the obligation of those requiring current investment to justify that investment. If you want the global economy to forgo over a $1 trillion annually, you need to make the case and make it strongly. You need to present a robust damage function. Otherwise, it just won’t happen. You may want to look up what the estimable Daniel Kahneman has written on global warming. (I dabbled in behavioral economics earlier in my career and while I moved on to a more markets based macro focus, I still regard him as one of the greatest living economists).
“Jim D,
I honestly don’t know what are are talking about.”
Don’t let it worry you David.
Self-evidently, nor does Jimbo.
So Mosher, at what point in time should the World have been induced to find K15 scientifically reliable without further caveat from the NOAA? The time of publication? Paris? Ever?
Does K15, as published, contain adequate notice of uncertainty sufficient for understanding by the Paris audience?
Should K15 have included notice of its lapses in Agency process and standards prior to publication, as part of the uncertainty necessarily contained in its conclusions?
“So Mosher, at what point in time should the World have been induced to find K15 scientifically reliable without further caveat from the NOAA? ”
1. We didnt take it as gospel truth. That is why we checked it.
Turns out They Improved the record.
2. I have no clue what you mean operationally by scientifically
reliable. Its a record, with uncertainty. Like all records. Their
Ocean now has more warming than ours. Not enough of
a difference to change any real science. A small technical
difference of interest of specialists.
Does K15, as published, contain adequate notice of uncertainty sufficient for understanding by the Paris audience?
1. Yes
2. There is ZERO evidence that anyone at paris even considered
K15.. In fact years of negotiating prior to K15 took no notice
of the pause. Quite the opposite. the pause which skeptics
found so compelling was ignored by at least half of scientists.
We published an editorial prior to K15 essentially making the
same point
Should K15 have included notice of its lapses in Agency process and standards prior to publication, as part of the uncertainty necessarily contained in its conclusions?
1. No.
2. The failure to archive data at the time of publication falls on the
journal to enforce.
Fundamentally I find the same thing here as I did in Climategate.
1. Processes… things like archiving, responding to FOIA, providing code,
being open and transparent are LACKING.
2. DESPITE these failings in Process NOTHING IN THE SCIENCE CHANGES.
The remedy is the same rememdy I suggested in Climategate.
1. Provide proper FUNDING for the improving processess and data archiving.
2. GET SCIENTISTS OUT OF THE JOB OF CONTROLLING THEIR DATA.
In industry we had data custodians and document control. I produced data and the custodian or librarian is in charge of filing it, and maintaining it, and distributing it.
” the pause which skeptics found so compelling was ignored by at least half of scientists” is an indicator that somewhere near 50% of ‘scientists’ are quite obviously incorrect on this issue.
> I would hope that formal CDRs would get folks to stop their claims
of fraud.
I would not. Not at this moment, where it appears that getting a seart at teh Donald’s craps-shooting table is on scientific activists’ agendas.
I can still Hope.
Still the day will come when the new commerce department has to
OWN the noaa data and processes or Take it down.
And they have already argued that our fear they will take down data is paranoid.
So that means.. someday.. they must OWN IT. It will be Trumps data.
Branded with a big gold gaudy T
They can
A) take it down and prove our concerns correct.
B) Leave it as is, and prove they dont really care.
C) Redo the science and find out that the answer doesnt change
even when you do all the paperwork
Other paths?
Honest question
> Other paths?
Stop funding the project.
Hire private corporations to redo everything.
Wonder what happens with democracy when so-called scientists contradict their POTUS.
Officially name Karl a data manipulator.
Create a commission led by Mr. T to rewrite the conclusions of each scientific report.
Look into teh Donald’s heart instead.
From infinite power comes infinite possibities.
“Other paths?”
D) Get Steven Goddard to re-do the science.
other paths…private industry is already involved as a private industry boat took over for part of the el niño buoys run when the govts climate ship was pulled off line in 2012
Well, show me that Karl’s bias was not a result of noticing these other datasets would be more consistent if he pressed his thumb this way.
There comes a question of circular reasoning when methods aren’t applied as expected. Someones intuition or bias might actually be borne out by the method, sure. But the bias needs to be separated from the science. Methods help that. You don’t know the extent of bias if the methods aren’t followed.
This is great! Shallow as it is, for me this is vindication after years of being laughed at and called names for being a ‘Climate denier’ in spite of citing studies by so many scientists and questioning the constant yearly trend of ‘adjustments’ made to the temperature data NOAA kept posting regularly.
I can ‘throw’ this article in the faces of the self-righteous GW believers out there.
And having said that……
Now let’s try to get our taxpayer money back for this embarrassing farrago of nonsense these so-called ‘scientists’ have foisted on us.
This ‘scam’ cost billions to perpetrate and it came out of all our pockets.
Let’s also get the ‘ringleaders’, who pushed this BS on us. I am sure there are a lot of politicians involved here ( and what a great way of winnowing the grain from the chaff here) and we should start hiring forensic accountants to find out the culprits (starting with the premier suspect—- the infamous Al G.)
Follow. The. Money.
I thought the article was rather dense, and seemed to argue that scientists had not rigorously kept records of their data and had also used weak statistical methods. But others in the comments are saying that subsequent studies nonetheless support the original data. Still, you think this is clear-cut evidence to “throw” at “believers”? I’m interested in why you think so.
I would ask the question that if the addition of ship data, the use of an “unclean” data set and relaxation of significance level made no difference, why did this paper have to be written at all. It sounds like the pre-existing data already had eliminated the “pause”. Where is that paper?
“scientists had not rigorously kept records of their data and had also used weak statistical methods”
He’s complaining that they didn’t archive records (on time) according to a protocol that he favors (and seems to have invented). As to statistical methods, he gives no details, and I see no evidence that he knows anything about statistics.
“The ‘whistle blower’ is John Bates who was not involved in any aspect of the work. NOAA’s process is very stove-piped such that beyond seminars there is little dissemination of information across groups. John Bates never participated in any of the numerous technical meetings on the land or marine data I have participated in at NOAA NCEI either in person or remotely. This shows in his reputed (I am taking the journalist at their word that these are directly attributable quotes) mis-representation of the processes that actually occurred. In some cases these mis-representations are publically verifiable.” – Peter Thorne
http://icarus-maynooth.blogspot.com.au/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
stove piped that’s a good word …nobody talks to anybody
SIB – This stale thread has suddenly burst back into life with much intriguing info from your good self!
Can you elucidate further on the NOAA “stove pipes” for example?
David Rose? Seriously? More “shoot the messenger” ad hominems against climate scientists, it gets boring after a while.
It must really be unsettling for you and others. Not just for these revelations but knowing that the big hammer will be coming down in the future. I don’t envy any of you.
Big hammer…
http://i.dailymail.co.uk/i/pix/2010/11/08/article-1327845-0BF704CC000005DC-414_468x286.jpg
Seriously Ceresco? “Revelations”? More like “Fabrications” on Mr. Rose’s part!
You’ve read about the baseline “confusion” I take it?
Reblogged this on I Didn't Ask To Be a Blog.
Dr. Bates, this has to be one of the most informative posts I’ve read here.
It also goes a long way in restoring my faith in scientists and our governmental science agencies. I still highly value my time as an intern with the USGS.
“In the fall of 2012, the monthly temperature products issued by NCDC were incorrect for 3 months in a row [link]. As a result, the press releases and datasets had to be withdrawn and reissued.”
What else was happening in the Fall of 2012????
a Benghazi cover-up….
and a Presidential election.
fall of 2012 the govts only ship solely dedicated to climate studies was pulled off line due to across the board budget cuts
“Berkeley Earth and HadCRUT that are relatively independent of the NOAA data sets, that agree qualitatively with the new NOAA data set. However, there remain large, unexplained regional discrepancies between the NOAA land surface temperatures and the raw data. ”
Judith.
. What Portion of HadCRUT data is taken from NOAA? In your study of this
what portion? 10% 20%? 50%? 90% 95%? how did you figure that?
What Portion of Berkeley earth land data is taken from GHCN-m version 3?.. how did you figure that?
What regional discrepancies are you referring to? What is large?
What do you mean by unexplained? what would constitute an explanation for you?
These are real questions. Since you express some certitude about these isssues an answer would be cool.
Ah yes, Berkely. Bastion of free speech and integrity.
As a libertarian and huge fan of Milo… I’ll say #Si
Judith – The Berkeley Earth and HadCRUT data are not relatively independent from the NOAA data sets. I will document this today in several tweets. They all draw from mostly the same raw data while in the case of the Berkeley analysis, the added sites are mostly in the same geographic area. Even Phil Jones has confirmed this with respect to NCDC, GISS and the HadCRUT data.
Roger Sr
“The Berkeley Earth and HadCRUT data are not relatively independent from the NOAA data sets”
The point is that they use HADSST3, and are not affected by this adjustment to ERSST.
In the 14 years at NCEI you never learned that Tom Karl always wins.
So, Mosher’s here running intereference?
One would think, in a case of data transparency like this, a different approach would have been seen.
Huh?
We Checked his work. That MEANS we didnt trust it.
You checked his work to be sure there was the right level of warming. After that, anything goes. Hardly proper, skeptical review of something that claimed to be a serious scientific discovery.
You checked his method and values, did it bother you that you were digging through a pooh mound? Verify all you want, the fleet data is not scientificly valuable.
I want to point out that all of the surface data sets over land suffer from i) a systematic warm bias associated with using minimum temperatures in the construction of trends and I) in blending non-spatially representative sites with good sites. This includes the BEST analysis as both of these issues directly affect the raw data. Maximum temperature is a better metric (although as the appropriate metric to diagnose global warming, ocean heat content change should be used).
Factored on top of that issue is the so-called homogenization issue which is very much a black box,
That there is a divergence between surface and lower tropospheric temperature trends supports the conclusion of warm bias in the surface temperature trend assessments. Warming has been occurring but it is more muted than claimed by Tom Karl et al.
An update on this subject will be given by Professor Dick McNider in next week’s Santa Fe meeting.
Roger Sr
” in blending non-spatially representative sites with good sites. This includes the BEST analysis as both of these issues directly affect the raw data. ”
1. There is no difference between “good” sites and other sites. That is,
I have tested multiple groupings of sites and multiple criteria for judging a site “good” or ‘not good’. No differences.
2. Did the same thing with spatially representative sites. Also, no difference.
Please have Dr McNider Provide his full data, code, and CDR. Otherwise we shall have to weigh his thumbs.
Translation; No matter what data you give us, we will give you the same answer.
I seem to remember Michael Mann having the same success with tree ring proxies. No matter what data you put in, he got the same results.
Post-normal statistics is funny that way.
Translation: whatever tests you may wish to use against any dataset, contrarian concerns will be raised.
Because, audits never end.
All this Wandering in the Weeds is fun and games, Mr. Mosher. But none of it alters the fact that IPCC climate models are bunk and unfit to fundamentally change our society, economy and energy systems.
And: https://bobtisdale.wordpress.com/2016/01/20/summary-of-the-oddities-in-noaas-new-pause-buster-sea-surface-temperature-product/
Go buy a life.
Steven Mosher, have you read the paper “Analysis of the impacts of station exposure on the U.S. Historical Climatology Network temperatures and temperature trends”? Roger Pielke Sr was a co-author.
The paper found no affect on average temperature trends from differences in quality of station siting.
If that was you found in your personal investigation then they confirmed it.
But read this line,
“The best‐sited stations show essentially no long‐term trend in diurnal temperature range, while the most poorly sited stations have a diurnal temperature range trend of −0.4°C/century.”
Did your investigation check for this?
See https://pielkeclimatesci.files.wordpress.com/2011/07/r-367.pdf
‘Roughly half of the available energy is returned to the atmosphere via latent heating on the wet day, with a maximum value at noon of ~350 W m-2. In contrast, the maximum latent heat value on the dry day is only 50 W m-2’
It seems very possible that there is a moving feast of drought and precipitation artefacts in the surface record. There is as well an ENSO artefact in the annual averages. The perils of over smoothing data. I think both of these contributed to last years ‘record’.
Well, what about the HAD data? Can anyone help me with this problem? In 2010 Phil Jones had an interview Q&A with the BBC and listed the warming trends from 1850 to 2009. This during their Climategate fiasco.
First trend was 1860 to 1880 0.163 c/ decade
Second trend was 1910 to1940 0.150c
Third trend was 1975 to 1998 0.166 c
Fourth trend was 1975 to 2009 0.161 c.
But now using the York uni tool the trends are——-
1860 to 1880 0.113 c/dec
1910 to 1940 0.129 c/dec
1975 to 1998 0.172 c/dec
1975 to 2009 0.188 c/dec
Why have the two earlier trends dropped and particularly the first trend 1860 to 1880 has dropped from 0.163 c to 0.113c ?
I’m using HAD 4 L&O, but there is a global HAD 4 Krig and that shows a higher trend for 1860 to 1880 of 0.167 c.
Just for interest I checked the trend from 1910 to 1945 and found it to be 0.140 c/dec or higher than Jones’s second trend is now. BTW HAD 4 global Krig was 0.151 c/ dec for 1910 to 1945. What is going on?
Here’s Jones’s 2010 BBC Q&A link.
http://news.bbc.co.uk/2/hi/science/nature/8511670.stm
And here is the York Uni data-base tool. Note that Cowton etc allowed RSS V4 TTT but not UAH V6, but only UAH V 5.6. Of course RSS V 3.3 TLT included.
http://www.ysbl.york.ac.uk/~cowtan/applets/trend/trend.html
Can anyone help me with this problem? In 2010 Phil Jones had an interview Q&A with the BBC and listed the warming trends from 1850 to 2009. This during their Climategate fiasco.
First trend was 1860 to 1880 0.163 c/ decade
Second trend was 1910 to1940 0.150c
Third trend was 1975 to 1998 0.166 c
Fourth trend was 1975 to 2009 0.161 c.
But now using the York uni tool the trends are——-
1860 to 1880 0.113 c/dec
1910 to 1940 0.129 c/dec
1975 to 1998 0.172 c/dec
1975 to 2009 0.188 c/dec
Why have the two earlier trends dropped and particularly the first trend 1860 to 1880 has dropped from 0.163 c to 0.113c ?
I’m using HAD 4 L&O, but there is a global HAD 4 Krig and that shows a higher trend for 1860 to 1880 of 0.167 c.
Just for interest I checked the trend from 1910 to 1945 and found it to be 0.140 c/dec or higher than Jones’s second trend is now. BTW HAD 4 global Krig was 0.151 c/ dec for 1910 to 1945. What is going on?
Here’s Jones’s 2010 BBC Q&A link.
http://news.bbc.co.uk/2/hi/science/nature/8511670.stm
And here is the York Uni data-base tool. Note that Cowton etc allowed RSS V4 TTT but not UAH V6, but only UAH V 5.6. Of course RSS V 3.3 TLT included.
http://www.ysbl.york.ac.uk/~cowtan/applets/trend/trend.html
Wow, what a bomb shell.
“All software has errors”
Ethics seems to be at the heart of this essay and it’s big deal. Being open, accessible and transparent are really important to some people.
Does this mean we should expect that President Trump to provide us with a machine readable version of his tax returns? Why not?
Oh and don’t forget that for every regulation/rule you want to add to enforce the CDR you have to kill two other regulations (see executive order released 1/30/17).
Whistleblowers… plural… maybe more hatchet jobs to come… same answer… same answer… same answer…
David Rose accompanied his story with this totally dishonest graph:.
https://wattsupwiththat.files.wordpress.com/2017/02/pausebuster-dataset.jpg
The difference there between NOAA and HADCRUT is almost entirely due to the difference in anomaly bases (1961-90 vs 1901-2000). If you put them on the same 1981-2010 base, it looks like this;
https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/hadnoaa.png
A single number for the entire planet? Wow. What’s the error bar look like? How many dancing angels can we get on the head of that pin?
Never mind the big picture of corruption in NOAA, let’s just concentrate on one minor graph! You couldn’t make it up. Oh, NOAA just did.
Gavin has also posted a better NOAA:HadCRU graph matching baselines. You can see Blue (HadCRU) slightly better in top in the past and see Blue much more in bottom in recent years, making Rose/Bates’ point more clearly than the Rose graph with two different baselines. So does yours.
Funny how NOAA runs cooler early on and a bit hotter later.
Nick, the point about the graph is a bit unsubtly made. In all likelihood, the intent of the author is to say the record high anomalies noted in NOAA data are not seen in HADCrut, which, is true.
Intent of the author?
I’ll add mindreading to your resume
Look into David Rose’s heart instead.
#AlternativeFacts
Pingback: And That’s The Fake News, Folks. Now, Here’s The Fake Weather. – waka waka waka
Anyone taking bets as to whether Science will pull the paper???
Same answer.
The question amounts to “Is Science doing science any more?”.
I fear that we all know the answer to that.
Shub,
” a bit unsubtly made”,/i>
Indeed so. That caption read:
“The misleading ‘pausebuster chart’: The red line shows the current NOAA world temperature graph – which relies on the ‘adjusted’ and unreliable sea data cited in the flawed ‘Pausebuster’ paper. The blue line is the UK Met Office’s independently tested and verified ‘HadCRUT4’ record – showing lower monthly readings and a shallower recent warming trend.”
Unsubtle. And lies. The difference he shows is almost entirely due to the different anomaly bases,, which are of no significance at all.
Good point. I missed that.
Regardless, HADCRUT global anomaly values are lower than NOAA, so the point stands. Picking on a graph shows desperation.
> HadCRUT4 is different from the most closely comparable products (e.g. NASA GISTEMP and NOAA MLOST) in that no interpolation is performed.
https://climatedataguide.ucar.edu/climate-data/global-surface-temperature-data-hadcrut4-and-crutem4
Let’s assume Hadcrut4 is the best. After all, it measured an English triangle in the 1850’s and there’s no interpolation.
Fascinating. Do continue. Please.
Typo In Bates’s text; change “date” to “data” in:
and so K15 must have used date with some experimental processing
he was in a rush
Instead of looking at QA as filtering data to a best set, a practical person might view it as characterizing all of the data so as to have maximum use. For example, soft or qualitative data may tell you something–particularly with spatial. The same goes for processes flawed to various degrees.
The fact that NOAA, or any entity collecting data, has a QA program for its purposes does not mean that those processes and products are best for a third party user who may have more stringent or even less stringent requirements–requirements constrained by available resources, timing, goals, etc. Sometimes we by necessity must just go with what we’ve got in hand. That does not preclude later extension, revision or iteration.
–particularly with spatial (data).
yup
Yep. Climategate II.
Another beat-up.
Even if there were a real pause, what difference would it make?
The temperatures are up and the ice is down. So energy has been accumulating. Because we are stuffing the atmosphere with insulation.
Paperwork notwithstanding.
Yep. Even if the temperature is up, and going up, so what? What difference does it make?
Even if the temperature is up or down says nothing about the impacts. The fact that there is no valid evidence that global warming would be net damaging is the real issue. Without valid evidence that GW is net harmful, there is no justification of the belief that GHG emissions are harmful – and no evidence to support the 2C political target and belief it is dangerous.
Peter : +1000
If all Climate Change is undoubtedly a Very Bad Thing, it follows that there must have been a time when we had the Perfect Climate, from which any deviation was for the worse.
When was it? How do we know?
Coz to me it seems that in general a warmer greener world is a better place than a colder greyer icier one…
Maybe if I was a polar bear I;d think different…but there are only ~25,000 of them living in a very narrow ecorange. By contrast there are 7,000,000,000 humans living all over the planet in temperatures varying from -40C to +40C. (233K to 313K). We are a very adaptable species.
And try as I might I really can’t persuade myself that the existence of the planet – or of humanity – is in danger if the average global temperature changes from 287K to 289K by the end of this century…
So what is all the ‘flaming’ fuss about?
> . The fact that there is no valid evidence that global warming would be net damaging is the real issue.
Indeed. We do not have evidence of the future.
FREEDOM!
nor from the present or past Willard.
HONESTY!
“The temperatures are up and the ice is down.”
Actually that isn’t true. The temperatures in the ’30s where higher than now. The ice cover in the Arctic was also lower than now.
“Because we are stuffing the atmosphere with insulation.”
Again, not true. the atmosphere doesn’t work like a layer cake, where each layer insulates the layer above from the layer below. It works like a gigantic convection engine. Trying to measure the insulating effect of CO2 isn’t even on the right planet. That’s the planet the global models trumpeted by the IPCC work on.
On the real Earth, convection from low to high and from equator to poles is what drives the climate.
Mark:
““The temperatures are up and the ice is down.”
Actually that isn’t true. The temperatures in the ’30s where higher than now. The ice cover in the Arctic was also lower than now.”
Actually. no it is true and you are peddling an “untruth”….
No doubt you do not trust NOAA so here is JMA:
http://ds.data.jma.go.jp/tcc/tcc/products/gwp/temp/fig/an_wld.png
And Hadcrut:
http://www.weathercast.co.uk/uploads/pics/hadcrut4_graph_small.jpg
Is there any more real truth you’d like me to show you.
Or, as I suspect, all you want is the alt type.
The endlessly perpetuated myth of the “1930’s were warmer”.
There is no data-set that shows that to be the case globally.
“On the real Earth, convection from low to high and from equator to poles is what drives the climate.”
No, the sun drives the climate, or rather the proportion of it’s TSI that is absorbed.
That currently is greater than the LWIR emitted.
The excess is largely accounted for by that being stored in the oceans.
Convection is one aspect of weather, which is the process that moves the Sun’s energy around the Earth in it’s quest to exit to space.
To some degree this is failing. Because there is a radiative restriction on it doing so above that balance dictates. All the science since Fourier, Tyndall and Arrhenius tells us that. Empirical science and not up for argument.
What is, is to what degree global temps will stop at when balance is re-attained.
Oh, and also you may, or may not, care to show comprehensive evidence of ice being less over the entire Arctic basin in the 1930’s than in recent years.
See, that’s the funny thing. I remember when the data-sets DID show the 30’s being warmer then today. Hell, I remember when Hanson himself displayed graphs showing the 30’s warmer then today.
Your basic problem here is, despite what the Leftists believe about human psychology and how to twist it, most people don’t actually fall for all this constant ‘adjustment’ of the data. Once or twice, maybe. But do it time after time, and people on both sides soon get cynical. And that goes double for Mosher and his assertions that everyone on the Climate Faithful side get the same results. Because it’s TRUE, they do all line up again soon after every adjustment. Over and over and over again, no matter how much they change, everyone always ends up with identical results. And we’re supposed to all believe that THIS one is correct, no matter how different it is from the past. We’re apparently not even supposed to notice how much it keeps changing. ‘The adjustments don’t really change the trend.’ ‘The raw data actually shows more warming.’ ‘No, I won’t show you the data, you just want to find something wrong with it.’
I’m sorry, but some of us remember when Oceania WASN’T at war with Eurasia.
“Your basic problem here”
Your basic problem is that you don’t pay attention. Hansen showed graphs which showed 1930’s close to recent for ConUS, not globally. And even that is out of date. 2012 and now 2016 were way ahead.
https://data.giss.nasa.gov/gistemp/graphs/graph_data/U.S._Temperature/graph.png
The NOAA CDR of total solar irradiance was given as an example of a well-documented data set. Unfortunately, the data are deeply flawed and the methodology dubious, so the data are not usable and should be an embarrassment for NOAA. I discuss the issue here:
http://www.leif.org/research/EUV-F107-and-TSI-CDR-HAO.pdf and in shorter form here http://www.leif.org/research/EUV-Magnetic-Field.pdf
My conclusion:
• There is no support for a variable TSI ‘Background’
• The current Climate Data Record [CDR] is not helpful to Climate Research
• The CDR should not be based on obsolete solar activity data
• I expect strong ‘push-back’ from entrenched ‘settled science’, but urge [at least] the solar community to be honest about the issue
I took the shorter version route. In principle some sources are better than others – i.e. – isotopes better than sunspots.
In general, I look for what caused this high-resolution ENSO proxy. More salt is La Nina and rainfall is in Australia.
https://watertechbyrie.files.wordpress.com/2014/06/vance2012-antartica-law-dome-ice-core-salt-content.jpg
http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-12-00003.1
My disappointment will be extreme – :) – if it is not a terrestrial amplifier of solar variability. I can tell you what it isn’t – it is not an internal effect in a resonant system.
Yes, So Bates doesnt know that CDR doesnt mean the data is good.
A CDR means that boxes were checked.
A CDR is a quality audit. It isn’t the same as ticking a few boxes. It is about ensuring that approved and valid processes were used to create the output from the source. There are no sources of corruption or bias. If you can’t show these simple things, your paper has no quality. It is more likely to be politics than science.
Who is checking the checkers?
The skeptics who have completely failed to show there is anything wrong with K15… because they appear incapable of doing science, but are clearly great at doing despicable smear jobs.
Meanwhile, the great scientists at NOAA, the same patriots who charged toward Mt. Suribachi with my father’s leathernecks, have been hard at work improving the SST data… version 5 is in the works.
http://celebrating200years.noaa.gov/events/wwii/iwo_jima650.jpg
“but are clearly great at doing despicable smear jobs.” Just like wielding the $900M ‘counter movement’ of ‘dark money’ is utilized vs. anyone labeled as a skeptic.
All’s fair………?
Chastise your own tribe and credibility improves.
JCH, “The skeptics who have completely failed to show there is anything wrong with K15… because they appear incapable of doing science, but are clearly great at doing despicable smear jobs.”
Actually, K15 showed there was something wrong with with previous version of ERSST which were used to estimate mitigation costs of perhaps a trillion dollars per degree. Bates showed that K15 didn’t follow established procedure. Getting the right answer using the wrong methods excuse is getting tired doncha think?
Librarians don’t take volcanoes.
“Getting the right answer using the wrong methods excuse”
There is no evidence about wrong methods. There are claims about late paperwork.
Nick Stokes, “There is no evidence about wrong methods. There are claims about late paperwork.”
The whole issue is about not following the methods recommended by NOAA which would include having all paperwork reviewed prior to release. NASA, NOAA and other US agencies are notoriously anal about procedure.
Nick, Mosher and the rest are absolutely right. It doesn’t matter if proper procedure is followed, as long as the ‘right’ answer is gotten. And it must be the ‘right’ answer because now that someone has gotten it, everyone else will be able to get the same ‘right’ answer too. Just like what happened with Mike’s Hockey Stick.
See, it’s like what the Left call ‘Fake but Accurate’, but even better. You don’t have to just take their word for it. Give them time and they’ll eventually fabricate plenty of ad hoc justifications and ‘evidence’ to support what they knew was true long before they even started.
” they’ll eventually fabricate plenty of ad hoc justifications and ‘evidence’”
All purpose answer to everything. No thinking required. Just as well.
Right JCH,
Because we all know that working in a lab or an office is every bit as dangerous as climbing a mountain. Not to mention little things like machine gun and mortar fire.
Anyone who puts on the uniform, whether voluntary or not, has my respect. Try not to diminish it by fake comparisons. Unless Mike Mann is your idea of a hero. Him being in the trenches and all.
Speaking of voluntary – the Generation of Heroes has been replaced. Today’s generation is truly that Generation. The majority of those landing on Iwo were draftees. That does not discount their courage, their efforts or their accomplishments. The men and women who have deployed to Iraq, Afghanistan and places all over the world have done so of their own accord. Everyone a volunteer. Most especially those who joined after 2001.
Amusing, I dismissed the post as it seemed just another politically motivated trivial witch hunt. But now it seems to have descended into a superficial discussion of the relation of surface temps to catastrophic global warming. Or not.
The satellite record seems preferable going forward – https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-836613 – but it is, of course, a small part of the energy budget.
https://watertechbyrie.files.wordpress.com/2014/06/ocean-temp1.jpg
I think I plotted that right on the Global Marine Argo Atlas. Discuss why ocean temps declined over the past couple of years – or not.
“I think I plotted that right on the Global Marine Argo Atlas.”
If you wanted to show the OHC at the meridian of Greenwich (0°E or 360°E) from 70°S to 70°N you plotted it right. Anyway, nobody would discuss why ocean temps declined because one should consider the whole ocean, from 0°E to 360°E:
http://fs5.directupload.net/images/170205/lysgdzyi.jpg
And now: what to discuss??
Good – we can talk about why things happen.
The change in ocean heat approximately equals energy in less energy out.
https://watertechbyrie.files.wordpress.com/2014/06/carbom-dioxide.jpg
Maybe not.
https://watertechbyrie.files.wordpress.com/2014/06/solar-flux.png
That’s a possibility.
https://watertechbyrie.files.wordpress.com/2014/06/toa-anomaly.png
Net up is warming.
Even if there were some increase in greenhouse gas forcing – it can’t be seen against large natural variability.
Climate forcing results in an imbalance in the TOA radiation budget that has direct implications for global climate, but the large natural variability in the Earth’s radiation budget due to fluctuations in atmospheric and ocean dynamics complicates this picture.
http://link.springer.com/article/10.1007%2Fs10712-012-9175-1
We may also wonder why you think that short term variability has any significance for the cause.
Your wrote:”Discuss why ocean temps declined over the past couple of years – or not.”
The ocean temps climbed indeed in the last couple of years as one sees when one uses the data of Argo correctly. Guess why?
Oddly enough I already have
Data URL?
Code?
CDR?
This post is about maintaining standards..
Well Mosh – it started with a complaint about adjustments to the surface temps – which uses such archaic methodologies – and contain such large and variable artefacts that are completely neglected – that it all becomes moot. It goes on the usual climate memes around pauses based on the surface temps. If you are going to claim off topic – please start well before I got here.
There are better methods than irrelevant to climate surface records. The only valid comparison is between UAH and RSS if you really want to maintain standards.
And the boxes, being checked, show that an extra level of care was taken, including review by some of the top people in the NCEI group. This should be better data than unchecked, which is much of the point that Dr Bates is making. Are you pushing for less checking and formality?
I’d push for more use of the satellite data. And the use of surface data for local weather.
The lack of quality control within the “alarmist climate change” community has astounded me for years. I have never trusted any of the surface temperature data, and for good reason.
This revelation is as big as Climategate, and confirms the corruption we have known about for years.
Pingback: The faking of climate data before the Paris coference | The k2p blog
Pingback: The faking of climate data before the Paris coference | The k2p blog
Earth is retaining more energy with more CO2.
Predicted. Observed. Measured.
It’s simple. Really simple. Watch the temperatures go up. Ice vanish. Coral die. More serious weather events. Droughts. Fires, Floods.
You won’t be able to deny it forever.
Yes nature bats last.
Nature bats every time Mosher. It’s her game. We are fans in the seats.
“Watch the temperatures go up. Ice vanish. Coral die. More serious weather events. Droughts. Fires, Floods.”
Which of those events show that man’s CO2 is the cause, for if man’s CO2 is not the cause there is no need for political action?
Or is it there is more CO2 with retained energy? Sure you have the cause/effect correct? What about H2O? What about the sun? What about precipitation? How about tilt?
That anyone can think one item is “in control” is ludicrous.
Lots of theories.
The truth is out there.
Are you seriously suggesting scientists don’t study and understand the Sun, orbits, tilt … No climate scientist amongst 10,000?
Without the greenhouse effect Earth’s temperature would be not be +15C but -18C. How much water vapour would there be at -18C.
CO2 is the control and we have increased the level from 280 to 400ppm: 40%.
Any rational person would expect Earth to retain more energy and not be surprised thermometers measure rising temperatures.
“They looked at satellites, Buoys and Argo.What did they find? They found that Karl’s thumb was weightless.That is, they found that the independent data sets CONFIRMED the adjustments.’
So all the “best” climate scientists (pun intended). using all the best temperature data, denied for almost 15 years that there was any pause.
Then about three years ago, those same scientists, using those same data sets, admitted there was a pause, and spent their energy explaining why it didn’t matter (ocean heat content being a better proxy was the most popular). Even the AR5 saw a pause in the data.
With Obama’s last shot at a climate deal coming up, we get Karl 15 which assures us there was no pause, based on “improvements” in the data.
The same data that didn’t show a pause for 15 years, then suddenly showed a 17 year pause that had to be explained away, was all wrong. And all those “best” scientists were wrong.
OK. this is post normal science at its typical. No surprise there.
But now we are assured that the data in Karl 15 that proves all those stupid best scientists were wrong, agrees with the data on which they made their calculations. The “adjustments” that show no pause, are “confirmed” (sorry “CONFIRMED”) by the data sets that until Karl 15 almost everyone agreed showed a pause.
But hey. let’s talk about charts!
+1
+1. And don’t forget the computer of the guy that made the breakthrough accidentally had a complete failure and nothing can be recovered. That’s not odd at all. And the 8 scientists that worked on this with him didn’t think they needed to have any backups. Again, that’s not odd at all.
“Watch the temperatures go up. Ice vanish. Coral die. More serious weather events. Droughts. Fires, Floods.
You won’t be able to deny it forever.”
Yeah, let us ignore that coral dying due to CO2 increases is unproven, as is increases in more serious weather events, droughts, fires, floods etc.
One does not have to deny things that are not real.
I think the defenders of Karl’s paper are not addressing the point. The point being “killing” the pause / slowdown up to 2014, and hurrying to publish it without any quality to be ready on time to have a political effect.
So, it doesn’t matter whether other temp series agree with Karl in 2016 or not. The question is the behaviour of the “settled science”, and whether there was a pause / slowdown with N papers “explaining it”, or it was just a scientists’ dream. You have to choose; one, or the other.
” hurrying to publish it without any quality”
That is Bates opinion. But the modification to ERSST was basically implementing the 2011 paper by John Kennedy, which had the necessary data. That is not undue haste.
So, your choice is the hiatus was Settled Science in dreaming mode.
On Bates, his “opinion” seems to have quite a lot of supporting facts. I should dig in it, though. Any of them you can point being wrong? Thanks.
Plazaeme,
Beyond the formal fussing about his archiving procedures, it’s mainly opinion, and the facts are few and shaky. Do you know where is the “thumb on the scales”? He makes a fuss about the choice of 90% CIs. They are actually common enough, but for this case, they are clearly required, because they are what is used in the AR5 analysis that he is comparing with. See Box 2.2, AR5.
A lot of the rest is just nit-picking, eg:
“The data used in the K15 paper were only made available through a web site, not in digital form, and lacking proper versioning and any notice that they were research and not operational data.”
Website is of course digital, and the paper is research, so that notice is hardly needed. This guy has an almighty chip on his shoulder.
” The land temperature dataset used in the Karl study had never been processed through the station adjustment software before, which led me to believe something was amiss. “
Again opinion. Or unbased suspicion.
All the fuss about GHCN is also just pointless nitpicking. K15 was about ERSST, and the adjustments for buoys.
Thanks, Nick.
Let’s take this for example:
– “It is clear that the actual nearly-operational release of GHCN-Mv4 beta is significantly different from the version GHCNM3.X used in K15. Since the version GHCNM3.X never went through any ORR, the resulting dataset was also never archived, and it is virtually impossible to replicate the result in K15.”
Do you think it is right or wrong? Thanks again.
Let me show you where I am going, in case the “”virtually impossible to replicate” part is right. You may say the GHCN part is not relevant. Not relevant for what? K15 “was about” ERSST. The pause killing thing.
OK. So Karl had two choices. To publish something “virtually impossible to replicate”, or to wait until he had a better quality paper. He chose the former because the pause killing thing was OK enough. And you think this is not “hurrying to publish it without quality, to be ready on time to have a political effect”?
Plazaeme,
Karl’s paper is here. It is a short paper, and has just three figures with global trends. They calculate with ERSST 3b and ERSST 4. The paper says they used GHCN3 on both occasions. It was presumably the same version. Now GHCN V3 has changed very little in that time. I don’t know what he means by GHCN V3.X, and I don’t know if he does, since he also refers to V4, which is a very different product. But GHCN V3 is issued more or less daily, and each file comes with a distinctive date number. I am sure the file for the relevant date is available. In any case, GHCN V3 has changed very little in the last few years (except for new data). It is extremely unlikely that getting the wrong day version would change any of the published results. Again, it is all about varying ERSST, which makes by far the largest part of the global average.
I’d endorse Mark Richardson’s comment here. Rose attributes some remarkably ignorant talking points to Bates. It may be misquoting, but I’m strating to doubt it.
You are mixing many things, Nick. So, I guess you are not very sure K15 was replicable when published. If this is the case, what I say stands. If not, no.
Plazaeme,
Actually, I was wrong on one point – the new analysis uses GHCN V4. But it’s a standard issue file, easily obtained. The readme file in the archive (dated Sept 15) says exactly what the are and where they are in the archive:
“This directory contains the adjusted land station data and metadata used in the Old Analysis; data from GHCN-Monthly version 3.2.2.
ghcnm.tavg.v3.2.2.20150116.qca.dat.gz; Adjusted station data
ghcnm.tavg.v3.2.2.20150116.qca.inv.gz; Inventory of adjusted stations”
And for the new Analysis
“This directory contains the adjusted land station data and metadata used in the New Analysis.
tavg.v4.a.1.20150119.qca.dat.gz; Adjusted station data
tavg.v4.a.1.20150119.qca.inv.gz; Inventory of adjusted stations”
It’s possible that if someone wanted to replicate the day after publication, they might have had to ask for a copy. For generations of science, that is how it worked.
plazaeme, as you and I know, Karl is not replicable. The lead author admits this to Rose.
“Karl is not replicable. The lead author admits this to Rose.”
Nonsense. Whta he said to Karl was:
“He also admitted that the final, approved and ‘operational’ edition of the GHCN land data would be ‘different’ from that used in the paper’.”
He’s saying that the eventual product may not give exactly the same result (of course). But he has archived the data files he used. That’s what are needed for replication.
Shub –
=={ The lead author admits this to Rose. }==
I see that rather unskeptically, you take Rose’s editorial commentary at face value.
Nick, are you able to replicate Karl’s paper? Has anyone reproduced the same results? A simple yes or no. If yes, then Bates’ argument is specious. If not, then the paper is crap.
Thank you very much, Nick. I think you are making a good case defending your critique. I was doubting whether to publish this news. I won’t. I don’t see the cookies.
Thank you Nick for confirming the point. What Karl et al did amounts to: ‘we were writing code for this paper and at one point we got this cool result (politically). We went ahead and published it even though we knew the final results would be different.’
If you did this in a clinical trial (and people have done this), the results would be thrown out.
*But the modification to ERSST was basically implementing the 2011 paper by John Kennedy, which had the necessary data. That is not undue haste.*
So this is an acceptable scenario under which to violate NOAA procedural requirements? (You have a prior paper that had the necessary data etc.)
Take the “haste” out of the equation, wasn’t Karl et al. supposed to follow NOAA’s review procedure?
I ask, because in this case, the behavior in question (failing to follow protocol) leads me to ask what else Karl et al. took shortcuts on … whether or not “politics” was the reason?
My understanding is the review process is there for a very important reason … to minimize the occurrence of questions like these.
John Ratcliff, with 170,000 + pages of federal regulations someone must have thought they were important :)
Is it not also NCEI policy for employees, observed here in the breach?
Geoff
Dear Dr Bates,
I just read the Rose article that quotes you and I wanted to check whether you’ve been accurately represented.
1) You’re quoted: ‘They had good data from buoys. And they threw it out and “corrected” it by using the bad data from ships. You never change good data to agree with bad, but that’s what they did – so as to make it look as if the sea was warmer.’
But ERSST gives anomalies so it makes zero difference whether you put the ships down or the buoys up. That part is irrelevant and misleading. Have you been misquoted?
2) It also ignored data from satellites that measure the temperature of the lower atmosphere…Dr Bates said he gave the paper’s co-authors ‘a hard time’ about this, ‘and they never really justified what they were doing.’
This sounds like they’re talking about MSUs, which would be a silly way of measuring SSTs since there is almost no SST information in those channels. Were you referring to IR?
3) “there needs to be a fundamental change to the way NOAA deals with data so that people can check and validate scientific results.”
Here’s a validation using freely available data:
http://advances.sciencemag.org/content/3/1/e1601207
Argo, ATSR/AVHRR and buoys alone all say that ERSSTv3b was missing warming and agree with ERSSTv4. Independent sources validate ERSSTv4 and this reality is the opposite of what David Rose says so I’m concerned that your name and concerns about data handling are being used to push absolute bollocks about scientific results.
Yes, those quotes show a high level of ignorant skeptic talking points. But they sound a bit too sophisticated for Rose to come up with.
“But ERSST gives anomalies so it makes zero difference whether you put the ships down or the buoys up. That part is irrelevant and misleading”
While no ‘expert’, common sense suggests these anomalies are for grids, not individual instruments (obviously!). In this case, the (rapidly) changing composition of ships v buoys must surely affect some grid anomalies, pushing them upwards. Otherwise, why bother with this adjustment at all?
“In this case, the (rapidly) changing composition of ships v buoys must surely affect some grid anomalies, pushing them upwards. “
No. The anomaly base they actually use is in the past. It used to be 1961-90; it seems now to be 1971-2000. It actually doesn’t matter, as long as it is a fixed period. What changes the trend is the increasing proportion of buoys recently, which read slightly colder, and this difference is in no way affected by the anomaly base.
MR, your first argument was tried at the time Karl appeared, and failed for two reasons. Bothmstill apply. First, you leave the better data alone and adjust the lower quality data. Always, I was taught. Second, there is an important artifact of moving the buoy data up. Over the period in question, the ERSST went from only about 10%buoy (iirc) to over 90%. So the substitution of data type over time automatically produces a warming trend where in fact none exists, something largely verified by ARGO and which was ignored. Clever, but wrong.
MR,
The elephant is the inability of Argo to measure the changes it purports to measure. It is simply not possible to attain the claimed accuracy, when you do the correct procedures and include the natural variation of (say) temperature down a profile, with its dynamics, its stratification, its variable mixing properties, surface micro layer effects and so on. Given past work at attempting water bath stabilised temperatures in good laboratory surroundings, I’d hazard a guess that the best one can do, in the sense of all variation in a natural setting, as about +/-0.1 deg C for uses such as ocean heat content. Happy to be shown wrong. Show me the numbers. My mind is flexible towards improved knowledge. So far nobody I have read has gone far towards this holistic accuracy, some do not go past elementary stats precision calculations. (Easy for me to have missed some critical work, there is a large volume.)
Remember the lesson from Josh Willis et al –
” The recent cooling signal in the upper ocean reported by Lyman et al.
[2006] is shown to be an artifact that was caused by a large cold bias discovered in a small fraction of Argo floats as well as a smaller but more prevalent warm bias in eXpendable BathyThermograph (XBT) data. These biases are both substantially larger than sampling errors estimated in Lyman et al. [2006].”
Who is to say the floats that replace Argo will not cause it to get junked like Argo junked the XBT versions?
Geoff
MR says – “But ERSST gives anomalies so it makes zero difference whether you put the ships down or the buoys up. ”
It makes a difference to the calculation of error magnitudes.
Geoff.
Mornin’ all (UTC)
A quick scan suggests nobody’s mentioned this in here yet. Zeke’s take(down?) on Bates, Curry, Rose et al. (2017)
https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise
Also:
http://icarus-maynooth.blogspot.com.au/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
http://variable-variability.blogspot.com.au/2017/02/david-roses-alternative-reality-noaa-Karl.html
http://fromtheprow.agu.org/climate-science-data-management/#comment-18185
https://www.theguardian.com/environment/climate-consensus-97-per-cent/2017/feb/05/mail-on-sunday-launches-the-first-salvo-in-the-latest-war-against-climate-scientists
Pingback: John Bates: Climate Scientists Versus Climate Data | The Global Warming Policy Forum (GWPF)
Pingback: Climate scientists versus climate data | Climate Etc. | Cranky Old Crow
Judith
Do you have any comment on David Rose failure to put the two temperature series on a common baseline.
I mean for chrissake. You cant just remain silent.
Mosher
Forget Rose. You and Hunt and others are missing the overall. The story is not about what Rose may or may not have said right or wrong. And it is not about the particulars of what Bates said properly or not.
The public can’t understand the details. And they don’t want to get into the weeds.
The story and the 2nd and 3rd derivative of the story is that a whistleblower, from the inside and not just any whistleblower but one fro the epicenter of the climate establishment. This has more significance, not scientifically, but public perception wise than anything that Judith or Pielke or Lindzencan say. Bates is from the government.
This is going to be a seminal moment because of the headline value. Every skeptic, politician or otherwise, will get their 15 minutes of fame, again not because of the actual issues surrounding what Bates has said but rather who has come out from the shadows. The original story will get lost. The future stories will be the great divergence between what the climate establishment really knows versus what they they think they know. And that is Judith’s uncertainty monster.
Anyone who thinks this is about Rose or about the specifics of Bates statements doesn’t understand the dynamics that will overtake what is being discussed here. Talk about chaos theory.
“The public can’t understand the details. And they don’t want to get into the weeds.”
“This is going to be a seminal moment because of the headline value.”
Well, I guess that is an honest assessment. But I think you overestimate the headlines. Another slimy article by Rose in the Sunday Mail won’t stop the presses. And it isn’t a slow news month.
Do you have any comment on David Rose failure to put the two temperature series on a common baseline.?
He is a journalist, the Guardian uses cute little polar bears dying on ice floes, he is selling a message.
Yes it is wrong, tough as they say. Know you have backed up your email correspondence with Karl, including the order to destroy it all.
Pass the message to Zeke, head down mate but not in an email.
> Forget Rose.
The request was made to Judy, not David.
Judy handed David the story, so that excuse won’t wash this time.
John will sooner or later have to face the fact that he made unsubstantiated claims (talk about traceability) that already have been refuted, eg: “Insisting on decisions and scientific choices that maximised warming and minised documentation” when:
http://icarus-maynooth.blogspot.com/2017/02/on-mail-on-sunday-article-on-karl-et-al.html
But yeah, we can forget about Senior. We can always do.
Shub – Fancy bumping into you in here. We’ll have to stop meeting like this! Of course Steve Mosher’s point is not irrelevant.
Ceresco – Please forgive my tardy reply, but I’ve been rather busy today poking the House Science Committee in general and Lamar Smith in particular with a very long but hopefully still sharp stick. By way of just one example:
https://twitter.com/GreatWhiteCon/status/828302371155562502
Now THAT is what this is all about.
Irrelevant.
David Rose, writing for a general public, could have qualified his story in many more ways to the point where it was caveats and exceptions, not able to be digested by most of the public. Me wanted to show two time series with different trends. He did. The numbers on the vertical axis allow readers to compare trend differences. Rose does not state that they are there to give absolute values to the lines. In some ways it is like other authors adding instrumental data to proxy data and explaining the addition in small print elsewhere. Acceptable to some, not to others. A specialist argument that need not be put in a newspaper article for the public???
Geoff
The better the evidence the deeper the faith. ABC. Anything But Carbon.
The global community of scientists faces “Challenge by book-keeping”.
“However, the Act will be toothless without an enforcement mechanism. For that, there should be mandatory, independent certification of federal data centres.”
Within other areas of importance, ISO 17025 is the standard by which independent laboratories are accredited.
“ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories is the main ISO standard used by testing and calibration laboratories. In most major countries, ISO/IEC 17025 is the standard for which most labs must hold accreditation in order to be deemed technically competent. In many cases, suppliers and regulatory authorities will not accept test or calibration results from a lab that is not accredited.
…
In the U.S. there are several, multidisciplinary accreditation bodies that serve the laboratory community. These bodies accredit testing and calibration labs, reference material producers, PT providers, product certifiers, inspection bodies, forensic institutions and others to a multitude of standards and programs.” – Wikipedia
I think the necessary standards and mechanisms for accreditation are largely in place. It is more about developing the competence and attitude towards using and following these standards. In particular, I think accreditation would benefit the field of climate science.
Africa one fifth of the world’s land mass-
WMO-
“Because the data with respect to in-situ surface air temperature across Africa is sparse, a oneyear regional assessment for Africa could not be based on any of the three standard global surface air temperature data sets from NOAANCDC, NASA-GISS or HadCRUT4 Instead, the combination of the Global Historical Climatology
Network and the Climate Anomaly Monitoring System (CAMS GHCN) by NOAA’s Earth System Research Laboratory was used to estimate
surface air temperature patterns ”
Urban Data
WMO-
“The nature of urban environments makes it impossible to conform to the standard guidance for site selection and exposure of instrumentation required for establishing a homogeneous record that can be used to describe the larger-scale climate”
“estimate” and “makes it impossible” – yes you could say i have full confidence in land based data.
“The other concern that I raised following ClimateGate was overconfidence and inadequate assessments of uncertainty.”
“I questioned another co-author about why they choose to use a 90% confidence threshold for evaluating the statistical significance of surface temperature trends, instead of the standard for significance of 95% — he also expressed reluctance and did not defend the decision.”
That is another area where climate science fails to meet industry standards.
There actually exists an international guideline on expression of uncertainty – Guide to the expression of uncertainty in measurement. This is the only broadly recognized guideline on the expression of uncertainty. The following seven organizations* supported the development of this Guide, which is published in their name:
BIPM: Bureau International des Poids et Measures, IEC: International Electrotechnical Commission, IFCC: International Federation of Clinical Chemistry **, ISO: International Organization for Standardization, IUPAC: International Union of Pure and Applied Chemistry, IUPAP: International Union of Pure and Applied Physics, OlML: International Organization of Legal Metrology ”. Guideline is freely available from https://www.oiml.org/en/files/pdf_g/g001-100-e08.pdf
United Nations Intergovernmental Panel on Climate Change IPCC failed to identify and recognize this guideline. They made up their own in a hasty way. Their guideline is largely a joke called: “Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties”.
https://www.ipcc.ch/pdf/supporting-material/uncertainty-guidance-note.pdf
That is the IPCC document that tries to standardise subjective levels of confidence. Hard to believe that this kind of sub-standard activity within science is allowed and even promoted by United Nations.
https://climateaudit.org/2007/06/10/ipcc-ar4-guidance-on-uncertainty/
AR4 Guidance on Uncertainty. Pertinent comments by
Ken Fritsch, Roman N , John A and others … What’s
being assessed?
Edit, ‘Roman M’
Thanks for that link. I also have two articles on my site about IPCC´s «Guidance note on the expression of uncertainty»:
IPCC was misled by the InterAcademy Council (IAC) on “Qualitative expression of confidence”!
Both IPCC and it´s reviewer, InterAcademy Council, messed up on “Quantified measures of uncertainty”!
And anyone believing that the review of IPCC by InterAcademy Council was independent will be surprised by this article:
The IAC review of IPCC was not independent!
In addition to having failed on qualitative and quantitative expressions of uncertainty, being susceptible for noble cause corruption, InterAcademy Council seem to have been heavily involved with United Nations and IPCC – both politically, economically, personally and otherwise.
One example:
“The creation of InterAcademy Council had been requested in 1999 by the Secretary‐General of the United Nations in order to facilitate the best scientific input into global decision‐making.”
Reblogged this on WeatherAction News.
Factcheck:
https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise
The past work described here as done by Dr Bates should be regarded as likely to do much good, little bad, thankless, repetitive, necessary, not every senior person’s cup of tea. The system is better for it.
It would be interesting to learn if the certifications required prior to use of collated data were matched by requirements that by-the-book treatment of precision, bias, uncertainty – whatever the labels – had been completed to specified methods and reviewed and passed before release for public use.
Dr Bates, if you read this a comment would be lovely.
Other readers, please do not go out of your way to be personally nasty. The real importance here is getting the best possible data to the users. It is trivially about whistleblowers and their motivation.
Geoff.
Wow
If only these guys had adopted the UEA/CRU standards of data management all of this unpleasantness could have been easily avoided.
Dr Phil Jones could so easily have destroyed/lost/ shredded/eaten/had his dog eat/ all the relevant data in ‘an office move’.
Or, as its known in the trade
‘The Climategate Method’.
Pingback: Data-fifflet bakom Parisöverenskommelsen avslöjat av NOAA-forskare - Stockholmsinitiativet - Klimatupplysningen
Pingback: Climate scientists versus climate data « How to s..t on humans
Who prepared the possibly misleading Figure for David Rose’s Daily Mail article – which allegedly uses different base years for calculating anomalies?
When I use Nick Stokes trend viewer, I get 1.56 K/century (0.67-2.00 95% ci correcting for autocorrelation) for what he calls NOAAlo and 1.34 K/century (0.97-2.18) for what he calls HADCRUT (for 1/97-11/16). The difference is 0.22 K/century or about 0.045K for this 1/5 century period. The graph has horizontal lines drawn every 0.2K, but the difference in temperature rise between these two data sets isn’t clearly visible on the plot. The only thing clearly visible there is the fact that NOAA is warmer in all periods – the baseline problem.
During the “Pause” (roughly 1997-2013), one can find periods where the trend of each record is as rough as 1 K/century less than the long term average of 1.6-1.7 K/century (1.4-1.9) for the last 40 years. The confidence interval also included zero during the Pause. However, now that we have experienced three years of warming in a row culminating in the recent El Nino, the trend for the first and last 20 years of this period is essential the same as for the full 40 year period! Karl15 didn’t kill the Pause – the 15/16 El Nino did. Unless temperatures return to the level of the Pause years for the next five years or so, the Pause is not going to return – whether or not Karl’s adjustments are accepted or refuted. Both sides should recognize that the process of combining data from an evolving mixture of technologies for measuring ocean temperature into a composite global record is an inherently uncertain process. The +0.05 K/decade change in trend introduced by Karl15 was only important because it challenged a politically-important term: “The Pause”.
More than three years of monthly posts at WUWT (and elsewhere) on the Pause by Monckton and others (which have mostly ceased), have created the illusion in part of the skeptical community that the Pause had invalidated the concept of radiative forcing. It’s time for this idiocy to stop. The relevant issue is TCR and especially ECS. There is no chance they are 0 K/doubling and little chance they are 1 K/doubling. They could be half of what the IPCC’s models project. Even RSS and UAH show long-term warming, albeit 25% less than the global surface record.
Here is my reproduction of Rose’s plot, using the native baselines (which differ)
https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/drose.png
And here is the 12-month running average smoothed version:
https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/drose1.png
The effect of the baseline diference, and the lack of effect of the trend, is obvious.
In the interest of reproducibility, here’s mine
https://andthentheresphysics.files.wordpress.com/2017/02/noaa_hadcrut4.png
Something that wasn’t clear to me, though, is that I had to use the data from NOAA’s site, not the data from the Karl et al. archive (which seems to have a baseline of 1971 – 2000, so is actually below the HadCRUT4 data if the baselines aren’t reconciled). Is the data on the NOAA site the same as the Karl et al. data?
ATTP,
The traditional files posted at NOAA are the MLOST files, which used to be 1961-1990, but may be now 1971-2000. They are the ones that are directly calculated, and would be what is in the archive. The 30-year baseline is one that can be used for most stations. Then the results are converted to 1901-2000 for most reporting purposes. I’m not sure why; it sounds like a longer base would be better, but in fact what counts is the base when you first aggregate. Annoyingly, the MLOST data is now posted later than the the 1901-2000 conversion.
Nick,
Thanks. I take it that the data that can be downloaded from the NOAA site uses the corrections presented in Karl et al.?
Jim Hunt link has Zeke’ replot showing the similarity, problem is not the graph misrepresentation by David Rose but the fact that a respected and senior insider felt that the head of the department was glory seeking and did it in an unprofessional manner.
This adds credence to the Republican investigation and means those who delayed and denied on the e mails now face much stiffer punishment as the investigation can now steam ahead.
Okay, I’ve just been reading the NOAA page about what data is used, and it seems that the answer to my question is that it is using the updated dataset.
Nick, The trend difference is easier to see when you use regular gridlines. They make it easier for the eye to see vertical and vertical intervals.
Why did you leave them out here?
Geoff
Nick: I don’t care what your graph shows with noisy or smoothed data, which can easily fool your eye. Rapidly changes in temperature look closer. The linear trend times the period using your trend viewer says that NOAA rose 0.045K more than HADCRUT over this period. If you want to convince me that the answer is different, show me a plot of the DIFFERENCE between the two records with a linear fit.
Changed caption in DM article: “Although they are offset in temperature by 0.12°C due to different analysis techniques, they reveal that NOAA has been adjusted and so shows a steeper recent warming trend.”
Hmm David Bates backstory here:
http://icarus-maynooth.blogspot.com.au/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
John Bates, but – yes – that does appear to be an excellent rebuttal. Seems to suggest that a great deal of what John Bates has said (or has been quoted as saying) is simply wrong.
ATTP,
Which is a fair comment of yours now that readers have validated your tremendous power to discern the presence of evil writers.
Do you really think you added value to this discussion?
Geoff
Yes, John, not David. Might as well meld David Rose and John Bates together though. :D
Here is the text of Peter Thorne’s article. Worth reading the lot….
On the Mail on Sunday article on Karl et al., 2015
There is an “interesting” piece (use of quotes intentional) in the Mail on Sunday today around the Karl et al., 2015 Science paper.
There are a couple of relevant pieces arising from Victor Venema and Zeke Hausfather already available which cover most of the science aspects and are worth a read. I’m adding some thoughts because I worked for three and a bit years in the NOAA group responsible in the build-up to the Karl et al. paper (although I had left prior to that paper’s preparation and publication). I have been involved in and am a co-author upon all relevant underlying papers to Karl et al., 2015.
The ‘whistle blower’ is John Bates who was not involved in any aspect of the work. NOAA’s process is very stove-piped such that beyond seminars there is little dissemination of information across groups. John Bates never participated in any of the numerous technical meetings on the land or marine data I have participated in at NOAA NCEI either in person or remotely. This shows in his reputed (I am taking the journalist at their word that these are directly attributable quotes) mis-representation of the processes that actually occured. In some cases these mis-representations are publically verifiable.
I will go through a small selection of these in the order they appear in the piece:
1. ‘Insisting on decisions and scientific choices that maximised warming and minised documentation’
Dr. Tom Karl was not personally involved at any stage of ERSSTv4 development, the ISTI databank development or the work on GHCN algorithm during my time at NOAA NCEI. At no point was any pressure bought to bear to make any scientific or technical choices. It was insisted that best practices be followed throughout. The GHCN homogenisation algorithm is fully available to the public and bug fixes documented. The ISTI databank has been led by NOAA NCEI but involved the work of many international scientists. The databank involves full provenance of all data and all processes and code are fully documented. The paper describing the databank was held by the journal for almost a year (accepted October 2013, published September 2014) to allow the additional NOAA internal review processes to complete. The ERSSTv4 analysis also has been published in no fewer than three papers. It also went through internal review and approval processes including a public beta release prior to its release which occurred prior to Karl et al., 2015.
2. ‘NOAA has now decided the sea dataset will have to be replaced and revised just 18 months after it was issued, because it used unreliable methods which overstated the speed of warming’
While a new version of ERSST is forthcoming the reasoning is incorrect here. The new version arises because NOAA and all other centres looking at SST records are continuously looking to develop and refine their datasets. The ERSSTv4 development completed in 2013 so the new version reflects over 3 years of continued development and refinement. All datasets I have ever worked upon have undergone version increments. Measuring in the environment is a tough proposition – its not a repeatable lab experiment – and measurements were never made for climate. It is important that we continue to strive for better understanding and the best possible analyses of the imperfect measurements. That means being open to new, improved, analyses. The ERSSTv4 analysis was a demonstrable improvement on the prior version and the same shall be true in going to the next version once it also has cleared both peer-review and the NOAA internal process review checks (as its predecessor did).
3. ‘The land temperature dataset used by the study was afflicted by devestating bugs in its software that rendered its findings unstable’ (also returned to later in the piece to which same response applies)
The land data homogenisation software is publically available (although I understand a refactored and more user friendly version shall appear with GHCNv4) and all known bugs have been identified and their impacts documented. There is a degree of flutter in daily updates. But this does not arise from software issues (running the software multiple times on a static data source on the same computer yields bit repeatability). Rather it reflects the impacts of data additions as the algorithm homogenises all stations to look like the most recent segment. The PHA algorithm has been used by several other groups outside NOAA who did not find any devestating bugs. Any bugs reported during my time at NOAA were investigated, fixed and their impacts reported.
4. ‘The paper relied on a preliminary alpha version of the data which was never approved or verified’
The land data of Karl et al., 2015 relied upon the published and internally process verified ISTI databank holdings and the published, and publically assessable homogenisation algorithm application thereto. This provenance satisfied both Science and the reviewers of Karl et al. It applied a known method (used operationally) to a known set of improved data holdings (published and approved).
5. [the SST increase] ‘was achieved by dubious means’
The fact that SST measurements from ships and buoys disagree with buoys cooler on average is well established in the literature. See IPCC AR5 WG1 Chapter 2 SST section for a selection of references by a range of groups all confirming this finding. ERSSTv4 is an anomaly product. What matters for an anomaly product is relative homogeneity of sources and not absolute precision. Whether the ships are matched to buoys or buoys matched to ships will not affect the trend. What will affect the trend is doing so (v4) or not (v3b). It would be perverse to know of a data issue and not correct for it in constructing a long-term climate data record.
6. ‘They had good data from buoys. And they threw it out […]’
v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. The assertion that buoy data were thrown away as made in the article is demonstrably incorrect.
7. ‘they had used a ‘highly experimental early run’ of a programme that tried to combine two previously seperate sets of records’
Karl et al used as the land basis the ISTI databank. This databank combined in excess of 50 unique underlying sources into an amalgamated set of holdings. The code used to perform the merge was publically available, the method published, and internally approved. This statement therefore is demonstrably false.
There are many other aspects of the piece that I disagree with. Having worked with the NOAA NCEI team involved in land and SST data analysis I can only say that the accusations in the piece do not square one iota with the robust integrity I see in the work and discussions that I have been involved in with them for over a decade.
A rebuttal to this (and the others) is under preparation
The focusing on the Rose article plot when there are bigger fish to fry, so to speak, shows a tendency to distract.
I’ve seen Mann’s twitter feed. The Team is distraught.
Don’t tweet, but agree. Head of AGU put out a Sunday PR pledging to defend the consensus. Sure sign this news is hurting.
To paraphrase Obiwan Kenobi: I sense a great disturbance in the farce.
I don’t see the point in arguing minor adjustments. It’s a great diversion.
It ties you up talking trivialities that the public has not the remotest interest in.
There are bigger distortions. Ends for trends – and the annual circus of annual means. Trends should start in the mid-forties – and only monthly data and running averages makes any sense in the larger scheme of things. The spike at the end is a combination El Nino – and a drought artefact from a severe global drought. Not much to base such dire narratives on.
https://watertechbyrie.files.wordpress.com/2014/06/hadcru-temps-with-line.png
Just between us, the trend is less than 0.1 degrees C/decade.
Changes in the Pacific state cause multi-decadal – and much longer – cooling and warming. Which is why a mid-40’s start is more honest than starting after temps fell. This is mainstream science btw. Concentrate on your strengths and their weaknesses.
https://watertechbyrie.files.wordpress.com/2016/02/sst_anomaly_amsre_2008105.jpg
And El Nino seems set to a nosedive off a 20th peak of El Nino activity.
https://watertechbyrie.files.wordpress.com/2014/06/vance2012-antartica-law-dome-ice-core-salt-content.jpg
Will this end global warming? Don’t know – but this will put a crimp in its style.
Oh and stick with the hyper impressive satellite technology – http://www.drroyspencer.com/2010/01/how-the-uah-global-temperatures-are-produced/
RE, whether you are right or wrong misses the main point imo . The debate is no longer about science, even if it once was. It is about politics and ‘solutions’. Showing corruption of NOAA data for political purposes makes a political, not a scientific point.
I’d agree – but you’re playing on their turf.
The way you draw that black line on that chart shows acceleration.
But you knew that right?
It is just a line between points showing a more rigorous choice for endpoints. But you knew that didn’t you.
More rigorous than what?
A least squares regression line?
The method used by Foster and Rahmstorf?
So the trend is less than 0.1 C per decade but accelerating, well that’s a relief.
The educated layman.. or practicing scientists from other fields might look at these and the long list of similar scandals and wonder why ‘climate scientists’ spend so little time and energy making sure their work is demonstrably honest and can stand up to external scrutiny.
And why so much on attacking the whistleblowers who point out their ‘mistakes’.
Curious
For all those defending K15, including Mosher and Stokes, would you please answer as simple question: have you been able to reproduce the K15 results? If so, then your criticisms of Bates are well founded. If not, then you’re full of crap.
Look through Mosher’s posts here. The answer is yes.
I believe ZH says the same thing. He was able to reproduce the work.
Well then,,they must have archived the data and code, right? I noticed none of those three have answered the question. Mosher says they checked it but no where says reproduced. I don’t think it has been.
Judith, John et al – I have tweeted with documentation showing (even in Phil Jones’s words) that the NCDC, GISS and HadCRUT analyses draw from mostly the same raw data. I also tweeted on why BEST is not a true independent analysis.
Systematic bias and uncertainties that we have documented in our peer reviewed papers apply to all of these analyses.
My recommendation is to use tropospheric layer averaged temperatures (from satellite and radiosondes) for atmospheric trends of heat changes.
For global warming diagnosis, use ocean heat content changes, recognizing that the deeper ocean heating (i.e. below the long term thermocline) is mostly unavailable to affect weather on multi-decadal time periods). Indeed, heat that goes deeper into the ocean is not even sampled by surface temperatures.
These more robust data tell us that the climate system has warmed in recent decades, but that the heating is more muted than claimed using the global surface temperature trend (and that the climate models are predicting). The heating is also quite spatially variable as shown in the ocean heat content data with a significant fraction going into the Southern Oceans.
Roger Sr
Roger,
Any thoughts on this post by Peter Thorne?
The post by Peter Thorne glosses over the objections by Bates.
Superfluous generalities. Right up your alley.
Ken – My comments are with respect to the culture of arrogance that Tom Karl, Tom Peterson and Peter Thorne showed when I was on the CCSP 1.1 Committee and afterwards. I was seeking to quantify uncertainties and biases in the Chapter I was lead author on, but they shut this effort down.
What John Bates has done is to expose this culture based not on robust science, but on promoting an agenda. Regardless of one’s views on policies, the scientific method should not be hijacked as they have done.
My specific scientific comments are with respect to the land portion of the surface data set. Subsequent to the CSSP Committee and in response to the failure of that report to properly assess the surface temperature analyses, we wrote this article
Pielke Sr., R.A., C. Davey, D. Niyogi, S. Fall, J. Steinweg-Woods, K. Hubbard, X. Lin, M. Cai, Y.-K. Lim, H. Li, J. Nielsen-Gammon, K. Gallo, R. Hale, R. Mahmood, S. Foster, R.T. McNider, and P. Blanken, 2007: Unresolved issues with the assessment of multi-decadal global land surface temperature trends. J. Geophys. Res., 112, D24S08, doi:10.1029/2006JD008229. http://pielkeclimatesci.wordpress.com/files/2009/10/r-321.pdf
There was one comment on it
Parker, D. E., P. Jones, T. C. Peterson, and J. Kennedy, 2009: Comment on Unresolved issues with the assessment of multidecadal global land surface temperature trends. by Roger A. Pielke Sr. et al.,J. Geophys. Res., 114, D05104, doi:10.1029/2008JD010450. http://pielkeclimatesci.wordpress.com/files/2009/10/r-321b.pdf
with our response
Pielke Sr., R.A., D. Stokowski, J.-W. Wang, T. Vukicevic, G. Leoncini, T. Matsui, C. Castro, D. Niyogi, C.M. Kishtawal, A. Biazar, K. Doty, R.T. McNider, U. Nair, and W.K. Tao, 2007: Satellite-based model parameterization of diabatic heating. EOS, Vol. 88, No. 8, 20 February, 96-97.http://pielkeclimatesci.wordpress.com/files/2009/10/r-322.pdf
I respect David Parker (who i have met) that he engaged on this. However, no response from the NCDC group.
Regards
Roger Sr
Roger,
Thanks for the reply, but Peter Thorne’s post suggests that a great deal of what John Bates has claimed is not true. I don’t really see how personal issues you may have with some of those involved is all that relevant.
How does what John Bates has done advance the scientific method?
Ken,
Pardon the interruption but impression is that science analyzes trends. Dr. Pielke shows data point. Dr. Bates indicates a continuation of the trend.
P.S. Here are a few examples of their ridicule rather than scientific engagement
http://rankexploits.com/musings/2009/enemies-caught-in-action/
https://pielkeclimatesci.wordpress.com/2011/11/23/tom-peterson-of-ncdc-and-climate-science-baloney/
https://pielkeclimatesci.wordpress.com/2009/11/21/comment-on-the-post-enemies-caught-in-action-on-the-blackboard/
attp, “How does what John Bates has done advance the scientific method?”
I believe John Bates is referring to the NOAA/NASA method as in rules and regulation that are in place. Pesky things they are.
Actually, my issues transcend personality. The problem, in this case, is the abuse of power (Tom Karl, Tom Peterson, Peter Thorne) to defend their surface temperature data, rather than to engage in constructive scientific discussion. The post by John Bates exposes their behavior.
Unfortunately, this culture of behavior and groupthink has become systemic throughout the climate science community including the AMS and AGU. My son’s recent WSJ op-ed documents his experience in this community.
Roger Sr.
Roger,
Do you regard accusing an entire scientific community of groupthink is acceptable behaviour? If so, who has the right to make such accusations and against whom can they be made?
I have documented the issues with the AGU and AMS leadership in both articles, reports and weblog posts. As just one example, AGU EOS refused to publish the minority report on the AGU Climate Change Statement despite explicit policy to publish opinion comments.
NCDC leadership is part of this mindset as John Bates has now documented.
The objective assessment of climate is broken. This should concern all scientists regardless of your political views.
Roger Sr
Notice how Senior dances around “Peter Thorne’s post suggests that a great deal of what John Bates has claimed is not true.”
What auditing standard should I use to document it?
Danny,
But one should also try to determine the significance of a trend.
Ken,
Hmm. There are two distinctions, IMO. First that there is a trend, then second the significance. One doesn’t eliminate the other. An example might be ‘the pause’. Isn’t that fair? Then, to where the trend line follows.
The point which leads from this topic is that rules of conduct are followed. Should they not be going forward then are the rules ‘useful’? If they are followed one might presume they indeed are.
Danny,
One could conclude that a trend is not significant – i.e., that there isn’t really any evidence for one.
Attp
So are you saying that if there is no trend then it could be called a hiatus?
Tonyb
Tonyb,
I’m suggesting that maybe we should be careful of drawing strong conclusions based on a few data points, especially if we don’t have much confidence in the veracity of the data.
Sufficiently high standard for evaluation of a trend?
“Last night Mr Karl admitted the data had not been archived when the paper was published. Asked why he had not waited, he said: ‘John Bates is talking about a formal process that takes a long time.’ He denied he was rushing to get the paper out in time for Paris, saying: ‘There was no discussion about Paris.’
He also admitted that the final, approved and ‘operational’ edition of the GHCN land data would be ‘different’ from that used in the paper’.”
1.) Data NOT archived due to a ‘process’ which takes time. (Please note time frame from the so called ‘hiatus’ and date of paper). What was the need for speed? (Not questioning motive, only asking a question which IMO needs asking).
2.) GHCN land data ‘different’. Some suggesting Dr. Bates decision to publicize the lack of following protocol, yet not asking in what form the ‘difference’ takes.
But that’s from the source so may not be found to be ‘useful’.
Attp
I am just joking with you.
I am on holiday in Austria at present and this thread is more entertaining than the alternative, which is a TV show which appears to be a dog beauty contest… my vote is for ‘teddy’ at present, but that could all change.
I am intrigued as to how we can see what appears to be tribalism developing here, as several people have linked approvingly to an article by peter Thorne. He appears to be highly credible but is he more credible than the equally illustrious author of this article? The fact he is retired also lends credibility as people seem More able to speak out.
However, as ī don’t tend to believe that scientists are trying to hoax us all my sympathies also veer towards Karl. What a dilemma!
I Think it will be easier to concentrate on the dog show Instead
Tony
Here is one:
Thorne writes “v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. ”
Sure the buoy data was weighted heavily – AFTER it was adjusted against the ship intake data. Now I’m not taking issue with that decision, even though it sounds counter intuitive to me – taking the less reliable data set and adjusting the more reliable one to it. What I do take issue with is Thorne spinning it to make it sound as if Dr. Bates is making a false statement. To claim that “v4 … makes preferential use of buoys over ships ” is effectively an effort to misdirect, as no mention is made of adjustments to it prior to it being used.
You two should hang out Ken. Thorne sounds like your kind of guy.
Ken Rice and Willard appear to see Peter Thorne as a pillar of truth, honesty and the climate science way.
They might be right on the last one, but it is pretty clear to see Thorne bending truth and honesty to make his point. A real pillar he is.
Ken Denison, following up on your question:. Would there have been a ‘warmest ever’ in time for the Paris accord had Karl adjustments not been performed?
Yes. HADCRUT and BEST did not have “Karl adjustments” and reported “warmest ever”.
Yes, but within error of estimate. So really, not. Disappointing, NS.
Oh, and you missed their PR to same? Don’t take my word, take theirs.
You really are off your game today. Unsettled by Gates revelations.
Did you reproduce K15’s results Nick? If so did you archive the data and code?
“Yes, but within error of estimate. So really, not.”
No, really. But anyway, BEST was a clear record. HADCRUT and NOAA were actually very similar, despite one having buoy adjustments and the other not. They both were narrowly warmest, but only because 2015 had been such a warm year. So there is no doubt that one or other is warmest. Here is the graph of cumulative record:
https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/records.png
And JMA.
Gavin has posted a better NOAA:HadCRU graph matching baselines. You can see Blue (HadCRU) slightly better in top in the past and see Blue much more in bottom in recent years, making Rose/Bates’ point more clearly than the Rose graph with two different baselines.
HadCrut4, though improved over HadCRAPPY3, is still essentially a regional temperature series, and has no important point to make about the GMST.
Reblogged this on The Ratliff Notepad.
https://arstechnica.com/science/2015/11/congressman-claims-noaa-whistleblowers-told-him-climate-study-was-rushed/
Bright Bulb or … Scientific integrity in the balance
For nearly two decades, I’ve advocated that if climate datasets are to be used in important policy decisions, they must be fully documented, subject to software engineering management and improvement processes, and be discoverable and accessible to the public with rigorous information preservation standards. I was able to implement such policies, with the help of many colleagues, through the NOAA Climate Data Record policies (CDR).
* John Bates, Climate Etc.
Robert L. Bradley, Jr.
robbradley58@gmail.com
713-304-8942
When they start peeling this onion it gets very very very ugly.
Climate “Science” on Trial; Data Chiropractioners “Adjust” Data
https://co2islife.wordpress.com/2017/01/29/climate-science-on-trial-data-chiropractioners-manipulate-data/
Consensus is more Con and NonSense than Science
https://co2islife.wordpress.com/2017/01/29/climate-science-on-trial-the-consensus-is-more-con-and-nonsense-than-science/
https://co2islife.files.wordpress.com/2017/01/1b1a8-jo_nova_ipcc_consensus_vote.png
Pingback: Because science! – Ordinarily Skeptical
Note how Judith Curry is always Judy and Roger Pielke Sr is Senior.
Professor Lindzen gets it right when he questions whether climate science data is even designed to actually help us answer questions–
meanwhile: https://science.house.gov/news/press-releases/former-noaa-scientist-confirms-colleagues-manipulated-climate-records
Thanks Frank, music to the ears, would not like to be Karl any time in the next 2 years
“Once the CDR program was funded, beginning in 2007, I was able to put together a team and pursue my goals of operational processing of important climate data records emphasizing the processes required to transition research datasets into operations (known as R2O). Figure 1 summarizes the steps required to accomplish this transition in the key elements of software code, documentation, and data.
“Unfortunately, the NCDC/NCEI surface temperature processing group was split on whether to adopt this process, with scientist Dr. Thomas C. Peterson (a co-author on K15, now retired from NOAA) vigorously opposing it.”
As long as John mentioned me I thought, perhaps, I would explain my concern. This is essentially a question of when does software engineering take precedence over scientific advancement. For example, we had a time when John’s CDR processing was producing a version of the UAH MSU data but the UAH group had corrected a problem they identified and were now recommending people use their new version. John argued that the old version with the known error was better because the software was more rigorously assessed. I argued that the version that had corrected a known error should be the one that people used – particularly when the UAH authors of both versions of the data set recommended people use the new one.
Which version would you have used?
Your concern would be valid but for three things:
1. SST adjustment method was based on a 2011 paper that reported great uncertainty in the result, although the central result was equivalent to Huang’s later paper introducing ERSST4 (the NOAA redo). Neither Huang nor Karl papers discussed that uncertainty. That borders on academic misconduct.
2. The Karl paper now provably did not follow the written archival requirements of Science. It should be retracted.
3. Your successors now say ERSST4?is flawed, will produce ERSST5, and it will show lower warming.
Dr Peterson, you have been caught out. You cannot defend your indefensible conduct by pointing to UAH or Bates rigid view on software validation. You used a land data set produced by software that was buggy. You used an SST methodology with high uncertainty and did not report that.
Good that you and Karl are retired, else your fired!
” based on a 2011 paper that reported great uncertainty in the result”
Kennedy reported a bias of 0.12±0.01°C. That is not great uncertainty.
“The Karl paper now provably did not follow the written archival requirements of Science. It should be retracted.”
You give no basis for that. But papers are not retracted for possible missed dates in archiving. There is a full archive now.
“Your successors now say ERSST4?is flawed, will produce ERSST5”
They don’t. There was always going to be an ERSST 5. Just like there was a 1,2,3,4.
As Ristvan said it just does not wash Thomas
Vigorously opposing accountability is a big no no.
Always.
Thomas Peterson: “Which version would you have used?”
Since you asked, I would have used the newer version of the UAH MSU data for internal research and analysis but I would not have published anything until this newer data had been certified by the CDR process. If any of the research results looked to have a potential impact on climate change policy (where perhaps trillions of dollars could ultimately be at stake), I would have expedited the CDR process.
Your concern about Bates rigidity on software validation is beside the point. There are three facts that make your and Karl’s actions indefensible:
1. The Huang ERSST4 methodology was based a previous 2011 paper. That paper reported a wide range of uncertainty; neither Huang nor Karl did. That borders on academic misconduct.
2. The data behind Karl is not archived as required by Science. The paper should therefore be retracted.
3. Your successors now say ERSST4 is faulty, and they are producing ERSST5, which will have a lower warming trend.
You rushed a politicized poor quality paper out to support Obama in Paris. Rep Smith subpoenaed NOAA based on whistleblower concerns, and that subpoena was stonewalled. Now we know why. And likely there were emails (probably now disappeared) between long time acquaintances Karl and Holdrun on this that explain the contempt of congress by NOAA?
Good that you and Karl are retired, else your fired.
“That paper reported a wide range of uncertainty; neither Huang nor Karl did. That borders on academic misconduct.”
Rud, you seem to be mindlessly repeating this. What is that “wide range of uncertainty” that you claim? Numbers, please.
The data IS AND WAS available
That is all Science Requires
Bates wants something different,, and Archive by his definition.
We want the data..
They gave us everything we required.. And skeptics had it too
That borders on academic misconduct.
Hilarious. Everything borders on academic misconduct.
Mosher says: “The data IS AND WAS available”
So you’ve reproduced the K15 results? Please post the link to data and code.
The sad truth is Mosher is well and truly in the black books with Karl and the AGW gang. They will never trust him and give him full access to any controversial data.
Hence he is able to make comments like
“The data IS AND WAS availableThe point is
The way Karl used the data IS AND WILL remain unavailable.
For ever.
That is what Science required ( and what science requires)
The method is lost forever Mosher, not the data, read what Bates said, carefully, and don’t dodge the real issue here.
Broke my New Years resolution saying this.
Thomas Peterson: This is essentially a question of when does software engineering take precedence over scientific advancement.
Whenever the “scientific advancement” depends on the output from the software.
I think the question you are really addressing is “When is the software considered reliable enough for claims and actions to be based on it?” There are examples, are there not, of “upgrades” to software introducing new errors, so newer can not be automatically granted a claim to superiority.
For example, we had a time when John’s CDR processing was producing a version of the UAH MSU data but the UAH group had corrected a problem they identified and were now recommending people use their new version. John argued that the old version with the known error was better because the software was more rigorously assessed.
Is that pertinent to this essay about this case? I think we can always take it for granted that no one is always right. If it is pertinent to this essay about this case, can you show us how?
According to Rt. Rear Admiral David Titley, former NOAA chief operating officer, apparently there is a proper process and procedure for any allegations of alleged ‘wrong doing’ at NOAA.
http://climatenexus.org/messaging-communication/current-events/climate-change-science-noaa-falsely-maligned-tabloid-spin
Bates neglected to follow that process.
Instead he chose to run to a Daily Mail trash tabloid ‘journalist’ with a known history of false claims, and a contrarian blog, to present his unrefereed opinions. Some of which have been refuted as provably false.
Right before a House Science Committee hearing to “Make the EPA Great Again” on Tuesday 7th Feb. (yes seriously, that’s what Lamar Smith called it)
https://science.house.gov/legislation/hearings/full-committee-hearing-making-epa-great-again
The House Science Committee and Lamar Smith tweeted about the Daily Mail trash tabloid article 6 times and did a Press Release. They used the Daily Mail trash tabloid piece s ‘confirmation’ (seriously) that NOAA scientists ‘manipulated climate records’ and were up to no good.
A trash tabloid piece as a ‘source’? No published science? No fact checking? No mention of other independent work that confirmed NOAA’s work? Really Lamar?
https://twitter.com/HouseScience
https://science.house.gov/news/press-releases/former-noaa-scientist-confirms-colleagues-manipulated-climate-records
The AGU commented on this too and supported the NOAA scientists:
“AGU believes that the merits of the Karl et al. (2015) should be and have been discussed in appropriate peer-reviewed scientific journals. We note that the main results of that study have since been independently replicated by later work. In the meantime, we will continue to stand up for the credibility of climate science, the freedom of scientists to conduct and communicate their science.”
https://fromtheprow.agu.org/climate-science-data-management
Why did Bates not follow the normal processes himself?
Why did the House “Science” Committee use a trash Tabloid hit piece as it’s only “source”?
Why?
I don’t think it’s a secret there is systemic bias within the climate science community. Why not focus the energy on this particular instance towards reproduction of results and quantifying and qualifying it?
One can only hope this will not be the only instance of someone in the background stepping forward to expose and help vett the information that has been conveyed over the years. I think we all agree that the truth is the ultimate goal regardless of stance. After all, temperature is such a simple and finite thing, no? ;-)
Just sayin…..
John Bates
Thanks for upholding the scientific method and working so hard to establish and maintain the integrity of the objective data.
What’s progress in requiring NOAA and the Inspector General to uphold and enforce the Information Quality Act for this climate data which appears to definitely come under “Important Scientific Information (ISI)?
What requests have been made to “correct” this information?
How can we collectively help to be most effective? (vs numerous easily dismissed requests)
How should be contact our legislators to fix this mess?
e.g. See:
NOAA also must maintain Information Quality, as it is subject to the Information Quality Act (Pub Law 106-554 via OMB’s guidelines.
National Oceanic and Atmospheric Administration Information Quality Guidelines
Office of Inspector General, Department of Commerce
House Committee on Science
Search on NOAA
@ ossqss
There’s temperature and then there’s attribution.
With respect to the old Leonard Nimoy show “In Search Of”, I am simply looking for authentication of either of the two items referenced and cannot find it to date. Any global game changing Paradigm Shift needs to be 100% supported without question. Not just an attempt of an educated opinion.
Former NOAA Scientist Confirms Colleagues Manipulated Climate Records
Feb 5, 2017 Press Release
Background: US House Committee on Science publications on “Karl NOAA”
Science Committee Oversight of NOAA’s 2015 Climate Change Study
Pingback: Heat’s On At NOAA As “High Level Whistle Blower” Exposes Data Fraud, “Scientific Integrity Breaches”!
Let me get this straight. Well known skeptic Judith Curry retired, willingly but disillusioned and early. Now runs a consultancy.
John Bates now runs a consultancy too. Retired from NOAA and nobody cares. Assumes the mantle of whistleblower. Bigly big show, brings the house down. Noticed at last.
Dennis
You make a good point that once people retire they can perhaps be more forthright with their opinions than when in work as they have little to lose
Tonyb
What opinion? Bates never had anything to do with the work.
http://icarus-maynooth.blogspot.co.nz/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw
Just another cunning stunt.
I told you all way last year that POTUS Trump’s impending election would pry the truth out of NOAA. And further that all this inconclusive bickering over the pause, clouds, attribution, little jimmy dee’s sanity, sensitivity and whatever would be rendered moot. Trump rules! Left loon alarmist climate policy drools, at least for the next 8 years. Then we will probably have POTUS Ivanka, if she wants it.
America seems to be saved since at least until we finally will bury the so-called Stadium Wave, Don Don:
https://twitter.com/DetonicsMKV/status/828107082368958464
Imagine all the money we can save if teh Donald signs a direct order eliminating second and third parties.
The AGU responds: 4 February 2017 CLIMATE SCIENCE & DATA MANAGEMENT
(Reference to Peer Review appears to be an effort to put the burden of proof on Karl’s challengers per NOAA’s Information Quality guidelines.)
OMB Peer Review Bulletin: NOAA
I was worried that things were mellowing out at Climate Etc.
Guess not.
I am having a nice day.
Thanks Judith.
Regarding Rose’s article, it is not likely that Karl affected anything at Paris, and even in the unlikely case it did, it is good they didn’t make any decisions on the now-known-to-be flawed hiatus. Thirty-year temperature trends never showed a pause, even pre-Karl, because there was a faster warming in the late 1990’s that canceled it out. Fifteen-year trends are inherently unstable because of the influence of the solar cycle among other things.
http://www.woodfortrees.org/plot/hadcrut4gl/mean:120/mean:240/plot/gistemp/mean:120/mean:240
Wrong when comparing temperature datasets to models to assess their validity. In BAMS 2009 NOAA said 15 years would invalidate them. In 2011 Santer published 17 years. Well, it has been 17 years and whether there is a hiatus like satellites show, or a slight warming per Karl, the fact is the models have now been invalidated by observation. By ~4x in the tropical troposphere per Christy, and still by 1.7x per Santer’s new paper with the erroneous tropical stratosphere correction. And about 2.5x for whatever GAST dataset one favors.
That means the model sensitivities are too high. Observationally based on several recent papers by a factor of about 2. Which means there is no C in CAGW.
This is the trend of 15-year temperature in degrees per decade. It is all over the place and therefore not a climate variable. Choosing pause periods is difficult to justify because a year or two later it isn’t. As recently as centered on 2000 the 15-year temperature had a trend of 0.3 C per decade. Skeptics never acknowledge this.
http://www.woodfortrees.org/plot/hadcrut4gl/mean:60/mean:120/derivative/scale:120/plot/gistemp/mean:120/mean:60/derivative/scale:120
The 30-year temperature has a much more robust trend staying within 10% of 0.17 C per decade since 1980.
http://www.woodfortrees.org/data/hadcrut4gl/mean:120/mean:240/derivative/scale:120/plot/gistemp/mean:120/mean:240/derivative/scale:120
“In BAMS 2009 NOAA said 15 years would invalidate them. In 2011 Santer published 17 years.”
As so often, no attention paid to what was actually said. BAMS 2009 was about the frequencies of 15 year pauses in ENSO-adjusted data. And Santer said that at least 17 years was required in order to detect human influence on troposphere temperatures.
JD, you did not even address the two papers cited. Do try harder.
Below you will see why I don’t trust satellites. I think when skeptics see the trend change between UAH V5.6 and V6.0 their most likely reaction is a wry smile. This is fake outrage.
Data was cobbled together from instruments not designed – current technology is massively improved.
Oddly the years after 1998 were adjusted much more than those before 1998.
From memory the newest purpose designed instrument took over in 2002. 21st-century data – from a variety if sources – is a revelation.
In April 2015, prior to Paris, UAH 6.0 was released. This cut the trend since 1998 by two thirds. This makes the Karl adjustment look like peanuts, and is equivalent to an admission of a major error by Spencer (again). However, now he is even further away from surface-based trends.
http://www.woodfortrees.org/plot/uah5/from:1998/trend/plot/uah6/from:1998/trend
Pingback: Sobre la cacareada “manipulación” de datos de temperatura de la NOAA | PlazaMoyua.com
“In experimental philosophy, propositions gathered from phenomena by induction should be considered either exactly or very nearly true notwithstanding any contrary hypotheses, until yet other phenomena make such propositions either more exact or liable to exceptions.
This rule should be followed so that arguments based on induction be not be nullified by hypotheses.” Isaac Newton
Climate shifts – duh. The next one is due within the decade. I predict it will be to more Pacific upwelling and cooler conditions. My hypothesis is that is driven by a solar trigger. As for the current trend – you don’t count your money while you’re sittin’ at the table…
https://watertechbyrie.com/2017/01/12/an-earnest-discovery-of-climate-causality/
The cool water anomaly in the center of the image shows the lingering effect of the year-old La Niña. However, the much broader area of cooler-than-average water off the coast of North America from Alaska (top center) to the equator is a classic feature of the cool phase of the Pacific Decadal Oscillation (PDO). The cool waters wrap in a horseshoe shape around a core of warmer-than-average water. (In the warm phase, the pattern is reversed).
Unlike El Niño and La Niña, which may occur every 3 to 7 years and last from 6 to 18 months, the PDO can remain in the same phase for 20 to 30 years. The shift in the PDO can have significant implications for global climate, affecting Pacific and Atlantic hurricane activity, droughts and flooding around the Pacific basin, the productivity of marine ecosystems, and global land temperature patterns. This multi-year Pacific Decadal Oscillation ‘cool’ trend can intensify La Niña or diminish El Niño impacts around the Pacific basin,” said Bill Patzert, an oceanographer and climatologist at NASA’s Jet Propulsion Laboratory, Pasadena, Calif. “The persistence of this large-scale pattern [in 2008] tells us there is much more than an isolated La Niña occurring in the Pacific Ocean.”
Natural, large-scale climate patterns like the PDO and El Niño-La Niña are superimposed on global warming caused by increasing concentrations of greenhouse gases and landscape changes like deforestation. According to Josh Willis, JPL oceanographer and climate scientist, “These natural climate phenomena can sometimes hide global warming caused by human activities. Or they can have the opposite effect of accentuating it.” NASA
This study uses proxy climate records derived from paleoclimate data to investigate the long-term behaviour of the Pacific Decadal Oscillation (PDO) and the El Niño Southern Oscillation (ENSO). During the past 400 years,
climate shifts associated with changes in the PDO are shown to have occurred with a similar frequency to those documented in the 20th Century. Importantly, phase changes in the PDO have a propensity to coincide with
changes in the relative frequency of ENSO events, where the positive phase of the PDO is associated with an enhanced frequency of El Nin˜o events, while the negative phase is shown to be more favourable for the development of La Niña events. Verdon, D. C., and S. W. Franks
(2006), Long-term behaviour of ENSO: Interactions with the PDO over the past 400 years inferred from paleoclimate records, Geophys. Res. Lett., 33, L06712, doi:10.1029/2005GL025052.
So a cool Pacific to 1976, warm to 1998 and a bit of a mixed signal since.
https://www.esrl.noaa.gov/psd/enso/mei/ts.gif
The energy budget feedbacks include cloud radiative effects. Limited satellite data says that this is significant. There are supporting ground based observations.
https://watertechbyrie.files.wordpress.com/2014/06/clement-et-al.png
The usual response from urban doofus hipsters is – naw this can’t be right. Some are playing catch up and spin. It can’t be very comfortable.
Oh and her’s the trends
http://woodfortrees.org/plot/gistemp/plot/gistemp/from:1944/to:1976/trend/plot/gistemp/from:1912/to:1944/trend/plot/gistemp/from:1976/to:1998/trend
Letter to: President Eric Davidson, American Geophysical Union
President Davidson
When 100 scientists attacked him, Albert Einstein said:
“If I were wrong, one would be enough”.
Isn’t science advanced more by model falsification than by consensus?
John P.A. Ioannidis has showed: “Why most published research findings are false.” PLoS Med. 2005;2(8):e124. pmid:16060722 and “Why Most Clinical Research Is Not Useful.” PLoS Med 2016; 13(6): e1002049. doi:10.1371/journal.pmed.1002049
Why should climate science be any different?
You assert: “all independent records now show that the past two years were the warmest years on record.”
Yet satellite temperature specialist Roy Spencer found that:
“Global Satellites: 2016 not Statistically Warmer than 1998” January 3rd, 2017 http://bit.ly/2iM6vFa
Please document how your statement is scientifically supportable within a 95% probability.
I find it remarkable that the president of the American GEOPHYSICAL Union would ignore the geological record documenting temperature declines since the Holocene Optimum. e.g. K. Gajewski (2016) reports:
“Most sites also show cooling during the past 3.2 ka.”
Quantitative reconstruction of Holocene temperatures across the Canadian Arctic and Greenland. Global and Planetary Change 128 (2015) 14–23 http://bit.ly/2l8klq6
Per Thomas Kuhn’s “The Structure of Scientific Revolutions” isn’t it very difficult to correct a scientific paradigm? By Nobel Laureate Richard Feynman’s high standard of scientific integrity, should we not bend over backwards to find any and every reason why a model is NOT true?
See Cargo Cult Science, Caltech 1974 http://calteches.library.caltech.edu/3043/1/CargoCult.pdf
John Bates has credibly documented serious failures at NOAA of not abiding by its own standards, (let alone those under the Information Quality Act.) http://bit.ly/2jS2QoD
Please lead the AGU in publicly examining all possible errors in NOAA’s temperature data and validating all its adjustments compared to the satellite and balloon records so we can have confidence in their veracity.
David L. Hagen
Posted Feb 5th http://bit.ly/2l8jPrN
“Simply put, if you’re attracted to ideas that have a good chance of being wrong, and if you’re motivated to prove them right, and if you have a little wiggle room in how you assemble the evidence, you’ll probably succeed in proving wrong theories right.”
https://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/
There is a great deal of very poor climate science. In the blogosphere it descends into climate memes learnt by rote at the feet of self appointed gate keepers. went back to realclimate recently. Naw – still crazy. As for hotwhopper – I am afraid to go back there.
Does any political moderate believe that there are not ideological underpinnings? The urban doofus hipster vision involves narratives of moribund western economies governed by corrupt corporations collapsing under the weight of the internal contradictions – leading to less growth, less material consumption, less CO2 emissions, less habitat destruction and a last late chance to stay within the safe limits of global ecosystems. And this is just in the ‘scholarly’ journals.
Economies are fragile – movements on markets can be fierce – recovery glacially slow sometimes. There are economic problems – but the problems are not intrinsic to capitalism. They were created by poor judgement. We blundered into it through stupidity. It is not difficult – however – to imagine scenarios in which markets are deliberately destabilised to hasten the end of capitalism. Creeping tax takes, overspending by government, printing money, keeping interest rates too low for too long, or too high for too long, taxing primary inputs, implementing market distorting subsidies – the scope is endless. These are suspiciously the objectives of global warming progressives – but let’s not call it a conspiracy. Climate science – and the weird fruit of it – is conflated with seriously misguided energy policy. Is that a coincidence?
As a bystander of little science expertise, but some experience in public policy analysis, here is a framing of the three major rebuttals I’ve seen to Bates’ claims.
(1) “On the Mail on Sunday article on Karl et al., 2015” by Peter Thorne at the blog of the Irish Climate Analysis and Research Units hosted by the Department of Geography at Maynooth University.
Most of this is classic “he said, she said” analysis. Bates made claims. Thorne says Bates is wrong. He does link to relevant information, which seems of little or no use to non-scientists (perhaps even to non-climate scientists).
(2) “David Rose’s alternative reality in the Daily Mail” by at Victor Venema (bio here) at his blog, Variable Variability.
His rebuttal makes two major points. First, the Daily Mail is a tabloid. Which is obviously correct. Second, that global average temperatures warmed during 1970-2000 — and there was a strong El Nino in 2015-16. He believes that this proves something, but is unclear exact what that is. Since his forte is study of variability, some detail on this point would be useful. Especially useful would be graphing the post-1950 time period, which is what the IPCC uses as the period dominated by anthropogenic warming.
(3) “Factcheck: Mail on Sunday’s ‘astonishing evidence’ about global temperature rise” by Zeke Hausfather of Berkley Earth, posted at Carbon Brief.
His rebuttal shows that NOAA’s news land surface record is similar to that of other major climate datasets, and that a new paper (on which he was lead co-author) confirms its sea surface data — “Assessing recent warming using instrumentally homogeneous sea surface temperature records” in Science Advances, January 2017.
If Hausfather is correct, this means that Bates’ claims are of interest to climate scientists, and esp. to NOAA and Science — most seriously about possible violations of best practices (and some rules) — but have little relevance to the public policy debate.
“Snow White” is mortified FM. Her words of wisdom are absent from your review article. Are you an expert on legal matters by any chance?
http://GreatWhiteCon.info/2017/02/climategate-2-falls-at-the-first-hurdle/#comment-217767
So, let’s just sit this one out and see who takes legal action
“The ‘whistle blower’ is John Bates who was not involved in any aspect of the work. NOAA’s process is very stove-piped such that beyond seminars there is little dissemination of information across groups. John Bates never participated in any of the numerous technical meetings on the land or marine data I have participated in at NOAA NCEI either in person or remotely. This shows in his reputed (I am taking the journalist at their word that these are directly attributable quotes) mis-representation of the processes that actually occured. In some cases these mis-representations are publically verifiable.”
– Peter Thorne, 2/5/17
http://icarus-maynooth.blogspot.co.uk/2017/02/on-mail-on-sunday-article-on-karl-et-al.html
Thank Heavens, David ppell has dropped it to tell us the unbiased truth! :(
I simply quoted Peter Thorne. Is he inconvenient?
Pingback: Links for 2-5-2017 – Karl Heubaum
Moreover, the GHCN software was afflicted by serious bugs. They caused it to become so ‘unstable’ that every time the raw temperature readings were run through the computer, it gave different results.
This sounds familiar – similar to the scandal of non-repeatability that has exploded in the life sciences, that is, medical drug discovery research especially involving molecular genetics. Attempts to replicate the highest impact papers fail more often than not. Scientists involved admit that their complex and sensitive experiments give different results with each run. This opens the door to cherry picking the results most to the liking of the author – a fraud that is easy to hide among the voluminous minutiae of the experimental method.
http://www.slate.com/articles/health_and_science/future_tense/2016/04/biomedicine_facing_a_worse_replication_crisis_than_the_one_plaguing_psychology.html
It’s a problem of massively inductive science. Assumption built on assumption, complex model on complex model, and data manipulation and steering of results becomes easier and easier with every layer of complexity.
Paradoxically, the real study of complexity is very simple – accept that it’s a chaotic-nonlinear system and analyse it accordingly. But politically progressive climate science refuses to do this – it sticks to inappropriate linearity and builds up monstrous analytical complexity where fraud can easily be hidden.
Just thought I’d repeat Mosher’s comment above:
It’s almost as though the truth were, well, inconvenient.
I didn’t see this figure until it was published. David Rose has now corrected the caption for the figure clarifying that two different baseline periods were used.
Thank you for the response.
It is very surprising that you continue such a close relationship with a journalist who repeatedly publishes such misleading information, whether carelessly or deliberately.
David Rose is an award winning investigative journalist https://en.wikipedia.org/wiki/David_Rose_(journalist)
Who has repeatedly written deeply misleading articles about climate change. As he did this time.
Your partnership with him is remarkable.
“Who has repeatedly written deeply misleading articles about climate change.”
Says the Leftist bedwetter who regards the specious alarmist drivel of Abrahams and Nutticelli of the Guardian as absolute authority on just about everything to do with climate “science”…
As a wise man once said, “you couldn’t make it up”!
Judith – Do you seriously expect us to accept that changing the caption and not the graph, without apology, satisfies clause 1.ii of the IPSO Editors Code of Practice?
http://GreatWhiteCon.info/2017/02/climategate-2-falls-at-the-first-hurdle/#comment-217736
“David Rose has now corrected the caption”
It is a very feeble change. The graph is still there, claiming in big print to show “Flawed NOAA data showing higher temperatures”:
http://i.dailymail.co.uk/i/pix/2017/02/04/23/3CD7C57C00000578-4192182-image-a-90_1486249374130.jpg
He claims in the revised caption that somehow “they reveal that NOAA has been adjusted”
Curryja:
“David Rose is an award winning investigative journalist https://en.wikipedia.org/wiki/David_Rose_(journalist)”
And, in regard to climate science at least, a very dishonest one….
https://blog.metoffice.gov.uk/tag/david-rose/
http://www.accuweather.com/en/weather-blogs/climatechange/uk-met-office-responds-to-misl/61017
Also just changing the headline on the graph is just not good enough.
Any honest reporter of facts would have had the two on the same base in the first instance, never mind not said why they were so far apart.
I know from experience that visuals tell the tale. “A picture paints …” etc.
99% of folk will see the graph. Assume “fraud” and move on.
The man is execrable and that you support him is likewise.
“And, in regard to climate science at least, a very dishonest one….”
You’ll be a bit of an expert on dishonesty in the climate science field, right?
Remind me, didn’t you used to work for the Met Office?
David Rose published graphs derived from public sources. If such graphs are to be compared, then some caveats need to be added whose words depend partially on the intended point of the comparison.
The initial incompatibility claimed about these 2 graphs arose from their authors, not from David Rose. Surely, there must come a point where investigative reporters must say “This is a rather complex comparison. How far down the explanatory track do I have to go to for the public to know that this climate special peculiar anomaly method of presentation has traps that we simply do not have the time nor space to explain in our media, nor attribute back to the original authors”.
It is not my view that David Rose erred. The error was with original authors who chose to present data that require too much detail to be placed in context each time they are used. And which are poorly classed by version and revision numbers and notes, and which can change overnight without much warning.
Geoff
“Remind me, didn’t you used to work for the Met Office?”
Cat:
Since you ask so nicely.
Yes.
And I am immensely proud of the fact.
I reckon I saved many military pilots lives over my career.
You?
Or Are you just proud of posting ad homs on worthless blogs?
Um, let’s see…
“Or Are you just proud of posting ad homs onworthless blogs?”
So in your view, Dr Curry’s blog is worthless…
“The man is execrable and that you support him is likewise
Er, you were saying?
Dear me, some people just have no self-awareness at all, do they Tony?
Why don’t you point out precisely which part of MY post constituted ad homs?
As far as I can see, I purely asked you two questions, so I’m intrigued as to how you managed to interpret them as insults…unless…
Hey VTG,
She lets you come here and insult her. Rose is a breath of fresh air in comparison to a twit like you.
Geoff Sherrington says:
“It is not my view that David Rose erred. The error was with original authors who chose to present data that require too much detail to be placed in context each time they are used.”
All published errors are the writer’s fault. Period.
Any climate journalist should know that different datasets often (and usually) have different baselines, and how to correct for them. In this case Rose did not, nor did he ask anyone.
David,
Climate journalist – does that require the same qualifications as a cartoonist? It sure does appear that way when one peruses the work of the majority of journalists who write about climate. Seth Borenstein, Chris Mooney, Scooter Nuccitelli (though in Scooter’s defense, he didn’t really start out as a journalist).
VTG and Mosher
In Rose’s figure the red and blue lines (Karl and Hadcrut) coincide at all the el Nino peaks. So just how can they be out of synch?
In any case they seem (to me) almost identical, just slightly displaced. Not really scandalous at all. The real issue is the serial hiking of especially Pacific SSTs by almost half a degree in the last decade. As Bill Illis puts it:
Bill Illis on February 5, 2017 at 7:05 am
So we move from ERSSTV2 to ERSSTV3 in 2009 and they adjusted the SST trend up by 0.3C. In V3 to V3b in 2012, adjustments of another 0.1C, The ERSSTV3b to ERSSTV4 in 2015 another +0.12C. That is 0.52C all together over just 6 years. And we don’t even really know what happened to the data in 2016 because noone knows where it comes from (some ships, ICOADs, where is the raw data).
It’s very interesting to look at who the House Science Committee Twitter account follows. https://twitter.com/HouseScience/following
Just sayin’ ;)
I respectfully disagree Geoff.
See clause 1(i) of the IPSO Editors’ Code of Practice op. cit.
Plagiarising Tony B(ii) – Every picture tells a story
Oh do cut some slack.
If climate journalists know about offsets and anomaly periods,as some claim they ought, surely the public should know also and so see abundant red herrings..
Or have said climate journalists failed in skills to educate the public about the anomaly method of expression of time series data?
Havebabthink about how many times a day you absorb material thastnyou knowbisvexpressed shorthand for any number of reasons. I want to read the nub of the material, not endless caveats, exclusions and so on.
Here, why not think in terms of every dayvreadingvrathere than a forensic dissection of a generalized message as if you were in a hunting party?
Geoff
Here’s a repeat of Bill Illis’ comment from the parallel thread on WUWT:
Bill Illis on February 5, 2017 at 11:47 am
If Karl was trying to come up with an accurate sea surface temperature dataset, he should have thrown out the inaccurate ship data instead.
But what he did in ERSST v3b was to throw out the satellite records followed up by throwing out the buoy trends in ERSST v4.
Does this sound like someone trying to get to an accurate record. Is this what a person in charge of a “National” data centre is supposed to be about. Is that what a person in charge of the world “Climate Data Centre” should be about.
We HAVE to go in and correct all of the data now. We are going to need forensic statisticians and prosecutors to do a proper job. I imagine there is an oath of integrity that Karl had to sign to be put in charge of so much of the world’s data records.
“But what he did in ERSST v3b was to throw out the satellite records followed up by throwing out the buoy trends in ERSST v4.”
Complete nonsense. The whole point of the adjustment was to make ships and buoys usable together. From Peter Thorne:
“v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. The assertion that buoy data were thrown away as made in the article is demonstrably incorrect”
Nick
I respect both you and Bill Illis, and I have no idea how to know which of you is right in this case.
Nick, how come you never replied to Bill Illis at WUWT?
I have been waiting for you there,afraid to address it in front of him?
“Nick, how come you never replied to Bill Illis at WUWT?”
You may notice that AW directed me to STFU. I am now on moderation there.
Nick
Sorry to hear you were closed down. I do not like echo chambers.
I wrote about sst’s some seven years ago
https://judithcurry.com/2011/06/27/unknown-and-uncertain-sea-surface-temperatures/
The data is staggeringly thin, incomplete, ramshackle and on the whole not worthy of being called scientific.
Do you seriously believe we have an accurate global record of sst’s to 1850 or so?
They begin to have some consistency and value from the 1950’s Onwards.
Would you agree with that date or go for the century longer dada as being scientifically viable?
Tonyb
Tony,
“Would you agree with that date or go for the century longer dada as being scientifically viable?”
I think longer. 1850 is a stretch. The thing is, SST varies fairly smoothly in both space and time, and in patterns which you can sort out with years of modern data. So you don’t need a high density of readings.
So Nick,
Are you now stooping to Thorne’s level of misdirection and dishonesty?
Adjusted the more reliable data set using the less reliable one and then weighted it? Saying the sole purpose of that exercise in statistics was to make the research easier rings hollow.
“Adjusted the more reliable data set using the less reliable one and then weighted it?”
There is a known discrepancy (0.12, Kennedy 2011). You have to adjust one or other to make them compatible. As has been said over and over, it makes no arithmetic difference which. The weighting, based on variance, is done regardless of the adjustment.
Since I ended up dropping statistical analysis in grad school, I’ll have to take your word that it doesn’t make a difference. As I have said I don’t have major heartburn over what the Karl research did. Just thought it didn’t sound logical. Not that it was improper. I do think that someone didn’t think it through very well though. Work for any decent sized company and HR will make you aware of the importance of avoiding even the appearance of conflict of interest, whether one exists or not.
I’d suggest a flanking move – and you’re not going to get there on a dead horse.
This is simple enough. Have I gone overboard on the title?
https://watertechbyrie.com/2017/02/06/2141/
The Irish Climate Analysis and Research Units blog did a critique of Bates’ claims and the Mail on Sunday article (already linked by Ceist (@Ceist8) above). They repudiate the claims in seven numbered points. I could see problems with two of them and tackled one, point 6. I tried to post my critique in a comment but fell foul of the word count. So I’m posting it here if that’s OK and they can come here and respond if they want to (I’ll let them know). Their blog is hosted by the Department of Geography at Maynooth University. Their critique of Bates is linked below, followed by my comment.
http://icarus-maynooth.blogspot.co.uk/2017/02/on-mail-on-sunday-article-on-karl-et-al.html?spref=tw&m=1
Regarding your 6 as copied and pasted below:
“6 [Bates] They had good data from buoys. And they threw it out […].”
Your response:
“v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. The assertion that buoy data were thrown away as made in the article is demonstrably incorrect.”
///
I note you are careful to say “as made in the article”. The analysis below doesn’t question the literal truth that raw buoy data was used in the paper. Everyone including Bates clearly knows that buoy data was used (after all, it had to be included as a starting point in order to adjust it upwards). The question is whether in the final analysis, the raw buoy data bore any resemblance to the adjusted buoy data. If not, then the raw data i.e. “the good data from buoys” was, for all intents and purposes, thrown out.
I clicked on the link to the ERSSTv4 paper you give in your response to point 6, quoted above. I presume your reference to preferential use of buoys over ships being weighted almost 7 times in favour is the 6.8 multiplication factor cited at the end of the second paragraph in section 2:
“The number of buoy observations was multiplied by a factor of 6.8, which was determined by the ratio of random error variances of ship and buoy observations (Reynolds and Smith 1994), suggesting that buoy observations exhibit much lower random variance than ship observations.” [From Section 2: Reconstruction Methodology- the full second paragraph of that section is reproduced at the bottom of this comment].
It’s of note that the 6.8 multiplication factor is introduced at the end of the paragraph and not at the beginning. At the beginning it says:
“buoy SSTA was adjusted by a mean difference of 0.12°C between ship and buoy observations (section 5).”
The specific section is 5C. I checked this section and there’s no mention of the 6.8 multiplication factor being applied during this stage. I was looking specifically for the “ratio of random error variances of ship and buoy observations” being applied during this adjustment process, thus triggering the 6.8 multiplication factor at the 0.12°C adjustment stage. In other words, there was no 6.8 multiplication factor weighting the 0.12°C adjustment. Indeed, section 5C lays out the adjustment process in four stages:
“Here the adjustment is determined by 1) calculating the collocated ship-buoy SST difference over the global ocean from 1982 to 012, 2) calculating the global areal weighted average of ship-buoy SST difference, 3) applying a 12-month running filter to the global averaged ship-buoy SST difference, and 4) evaluating the mean difference and its STD of ship-buoy SSTs based on the data from 1990 to 2012 (the data are noisy before 1990 due to sparse buoy observations).”
The only thing which might possibly refer to a multiplication factor here is in point 2), a “global areal weighted average”. It certainly isn’t the 6.8 multiplication factor mentioned at the end of the second paragraph in section 2. Besides the 6.8 factor is said to be based on the “ratio of random error variances” and nothing else which is why I was looking for that specific term in section 5C.
Therefore, the 0.12°C upward adjustment to buoy data was made before this same upward adjustment was subsequently compounded by the 6.8 multiplication factor. It therefore follows that your stated “preferential use of buoys over ships….weighted almost 7 times in favour” is, on closer analysis, a 7 times compounding of the upwardly adjusted 0.12°C buoy data. This means that, far from being weighted 7 times in favour of the original, unadjusted buoy data, it’s weighted 7 times *against* it (prior to SSTA averaging, see below).
The process continued as follows:
“The averaging of ship and buoy SSTAs within each 2° × 2° grid box was based on their proportions to the total number of observations. The number of buoy observations was multiplied by a factor of 6.8, which was determined by the ratio of random error variances of ship and buoy observations”. (Also from the second paragraph in Section 2).
In other words, the number of buoy SSTA data points in each 2° x 2° monthly bin was multiplied “almost 7 times” meaning that each and every buoy SSTA data point that had by now been adjusted up by 0.12°C was then reproduced 7 times over in each and every bin. This of course served only to increase the proportion of (upwardly adjusted) buoy SSTA data to ship SSTA data. Only then was the averaging of the two sets performed in each bin with the effect being that the increased proportion of upwardly adjusted buoy data had far more influence on that average than it would otherwise have done. That bumped the averaged result up by 0.06°C for the crucial post 1980 period. As stated in section 5c:
“However, the global mean SST is 0.06°C warmer after 1980 in