Response to critiques: Climate scientists versus climate data

by Judith Curry

Not surprisingly, John Bates’ blog post and David Rose’s article in the Mail on Sunday have been receiving some substantial attention.

Most journalists and people outside of the community of establishment climate scientists ‘get it’ that this is about the process of establishing credibility for climate data sets and how NOAA NCDC/NCEI have failed to follow NOAA’s own policies and guidelines, not to mention the guidelines established by Science for publications.

In this post, I go through the critiques of Rose/Bates made by NOAA scientists and other scientists working with surface temperature data. They are basically arguing that the NOAA surface temperature data sets are ‘close enough’ to other (sort of) independent analyses of surface temperatures. Well, this is sort of beside the main point that is being made by Bates and Rose, but lets look at these defenses anyways. I focus here more on critiques of what John Bates has to say, rather than the verbiage used by David Rose or the context that he provided.

The Data: Zeke Hausfather and Victor Venema

You may recall a recent CE post where I discussed a recent paper by Zeke Hausfather Uncertainties in sea surface temperature. Zeke’s paper states that it has independently verified the Huang/Karl sea surface temperatures.

Zeke has written a Factcheck on David Rose’s article. His arguments are that:

  1. NOAA’s sea surface temperatures have been independently verified (by his paper)
  2. NOAA’s land surface temperatures are similar to other data sets
  3. NOAA did make the data available at the time of publication of K15

With regards to #1: In a tweet on Sunday, Zeke states

Zeke Hausfather ‏‪@hausfath   ‪@KK_Nidhogg@ClimateWeave @curryja and v5 is ~10% lower than v4. Both are way above v3, which is rather the point.

What Zeke is referring to is a new paper by Huang et al. that was submitted to J. Climate last November, describing ERSSTv5. That is, a new version that fixes a lot of the problems in ERSSTv4, including using ships to adjusting the buoys. I managed to download a copy of the new paper before it was taken off the internet. Zeke states that v4 trend is ~10% lower than v5 for the period 2000-2015. The exact number from information in the paper is 12.7% lower. The bottom line is that sea surface temperature data sets are a moving target. Yes, it is good to see the data sets being improved with time. The key issue that I have is reflected in this important paper A call for new approaches to quantifying biases in observations of sea surface temperature, which was discussed in this previous CE post.

Regarding #2. Roger Pielke Sr. makes the point that ALL of the other data sets use NOAA’s GHCN data set. Zeke makes the point that CRUT and Berkeley Earth do not use the homogenized GHCN data. However, as pointed out by John Bates, there are serious problems with the GHCN beyond the homogenization J

Regarding #3. John Bates’ blog post states: “NOTE: placing a non-machine readable copy of a dataset on an FTP site does not constitute archiving a dataset”

Victor Venema has a blog post David Rose’s alternative reality.  The blog post starts out with a very unprofessional smear on the Daily Mail. He provides some plots, cites recent articles by Zeke and Peter Thorne. Nothing worth responding to, but I include it for completeness. The key issues of concern are in John Bates’ blog post (not what David Rose wrote).

The fundamental issue is this: data standards that are ‘ok’ for curiosity driven research are NOT ‘ok’ for high impact research of relevance to a regulatory environment.

Peter Thorne and Thomas Peterson

Thomas Peterson, recently retired from NOAA NCDC/NCEI, is a coauthor on K15.  He tweeted:

Thomas Peterson ‏‪@tomcarlpeterson 16h  Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.

Response: Well, in the new Huang et al. paper on ERSSTv5, it turns out that adjusting the ships to buoys results in a trend that is lower by 0.07oC. NOT exactly the same – in the climate trend game, a few hundredths of a degree actually matters.

In the comments on John Bates’ post, Peterson writes:

As long as John mentioned me I thought, perhaps, I would explain my concern. This is essentially a question of when does software engineering take precedence over scientific advancement. For example, we had a time when John’s CDR processing was producing a version of the UAH MSU data but the UAH group had corrected a problem they identified and were now recommending people use their new version. John argued that the old version with the known error was better because the software was more rigorously assessed. I argued that the version that had corrected a known error should be the one that people used – particularly when the UAH authors of both versions of the data set recommended people use the new one. Which version would you have used?

John Bates email response:

First, Peterson is talking about a real-time, or what we dubbed and interim CDR, not a long term CDR. So he is using incorrect terminology. In fact the UAH interim CDR was ingested and made available by another part of NCDC, not the CDR program. Of course, I never said, use the old one with the known error. But, what I actually did is check with the CDR program for what they did. And since, yes, this was fully documented I can go back and find exactly what happened as the trail is in the CDR document repository. The CDR has a change request and other processes for updating versions etc. This is done all the time in the software engineering world. As I recall, there was a very well documented process on what happened and it may actually be an example to use in the future.

So, Peterson presents a false choice. The CDR program guidelines are for non-real time data. We did set up guidelines for an interim CDR, which is what Peterson is referring to. So, customers were provided the opportunity to get the interim CDR with appropriate cautions, and the updated, documented CDR when it became available later. 

Peter Thorne is coauthor on both Huang et al. ERSST articles. From 2010 to 2013, he was employed by North Carolina State University in the NOAA Cooperative Institute for Climate and Satellites (CICS). He has never been directly employed by NOAA.

Thorne published a blog post On the Mail on Sunday article on Karl et al.  Excerpts:

The ‘whistle blower’ is John Bates who was not involved in any aspect of the work. NOAA’s process is very stove-piped such that beyond seminars there is little dissemination of information across groups. John Bates never participated in any of the numerous technical meetings on the land or marine data I have participated in at NOAA NCEI either in person or remotely. This shows in his reputed (I am taking the journalist at their word that these are directly attributable quotes) misrepresentation of the processes that actually occurred. In some cases these misrepresentations are publically verifiable.

Apparently Peter Thorne does not know much about what goes on in NOAA NCDC/NCEI, particularly in recent years.

Response from John Bates:

Peter Thorne was hired as an employee of the Cooperative Institute for Climate and Satellites NC in 2010 and resigned in may or June 2013. As such, Thorne was an employee of NC State University and not a government employee. He could not participate in government only meetings and certainly never attended any federal manager meetings where end-to-end processing was continuously discussed. As I discussed in the blog, my Division was responsible for running the ERSST code and the global temperature blend code from 2007-2011. We had begun more fully documenting that code including data flow diagrams and software engineering studies. In addition, my Division ingested and worked with the all the GHCN data and the ICOADS data. I developed extensive insight into how all the code ran, since I was responsible for it. Running of the ERSST and global temperature blend code was transferred to the other science Division in late 2010, prior to the arrival of Thorne at NCDC. Since I remained part of the management team the remainder of my time at NCDC/NCEI, I had deep insight into how it ran.

The key issue is this. John Bates is not a coauthor on any of these studies, and hence doesn’t have any personal vested interest in these papers. However, he is extremely knowledgeable about the subject matter, being the supervisor for the team running ERSSTv3 and the GHCN. He has followed this research closely and has had extensive conversations about this with many of the scientists involved in this research. Most significantly, he has collected a lot of documentation about this (including emails).

So this is not a ‘he said’—‘the other he said’ situation. Here we have a scientist that spent 3 years 2010-2013 at NOAA (but wasn’t employed by NOAA) and is coauthor of two of the papers in question, versus a supervisory meteorologist employed by NOAA NCDC for nearly two decades, that was formerly in charge of the Division handling the surface temperature data and the architect of NOAA’s data policies.

Regarding Thorne’s specific points:

  1. ‘Insisting on decisions and scientific choices that maximised warming and minimised documentation’

Dr. Tom Karl was not personally involved at any stage of ERSSTv4 development, the ISTI databank development or the work on GHCN algorithm during my time at NOAA NCEI. At no point was any pressure bought to bear to make any scientific or technical choices. It was insisted that best practices be followed throughout. The GHCN homogenisation algorithm is fully available to the public and bug fixes documented. The ISTI databank has been led by NOAA NCEI but involved the work of many international scientists. The databank involves full provenance of all data and all processes and code are fully documented.

Response: Thorne has not been on site (NOAA NCDC) for years and not during the final few years when this took place. 

  1. ‘The land temperature dataset used by the study was afflicted by devastating bugs in its software that rendered its findings unstable’ (also returned to later in the piece to which same response applies)

The land data homogenisation software is publically available (although I understand a refactored and more user friendly version shall appear with GHCNv4) and all known bugs have been identified and their impacts documented. There is a degree of flutter in daily updates. But this does not arise from software issues (running the software multiple times on a static data source on the same computer yields bit repeatability). Rather it reflects the impacts of data additions as the algorithm homogenises all stations to look like the most recent segment. The PHA algorithm has been used by several other groups outside NOAA who did not find any devestating bugs. Any bugs reported during my time at NOAA were investigated, fixed and their impacts reported.

Response:  Thorne left NOAA/CICS in 2013. As outlined in the original blog post, the concern about GHCN was raised by a CMMI investigation that was conducted in 2014 and the specific concerns being discussed were raised in 2015. I cannot imagine how or why Peter Thorne would know anything about this.

  1. ‘The paper relied on a preliminary alpha version of the data which was never approved or verified’

The land data of Karl et al., 2015 relied upon the published and internally process verified ISTI databank holdings and the published, and publically assessable homogenisation algorithm application thereto. This provenance satisfied both Science and the reviewers of Karl et al. It applied a known method (used operationally) to a known set of improved data holdings (published and approved).

Response from John Bates:

Versioning of GHCNmv4 alpha – So, after I sent my formal complaint on K15 to the NCEI Science Council in Jan 2016, I pressed to have my concern heard but sessions were booked. I pressed on at the end of one of the meetings and specifically brought up the issue of versioning in additional to not archiving. Russ Vose Chairs the Science Council and Jay Lawrimore, who runs the GHCN code, were in attendance. I made my argument that the version in K15 was in fact V4 alpha and should have been disclosed as such whit the disclaimer required for a non-operational research product. I said that the main reason for changing the version number from 3 to 4 was the use of ISTI data per what Jay Lawrimore had briefed. Moreover, plots of raw, uncorrected ISTI vs GHCN 3 on the ISTI page (will find link after I send these thoughts) show that there are 4 decades in the late 1800 and early 1900s where there is a systematic difference of 0.1C between the two. The reason for this has to be explained before the data are run through the pairwise, and so since there is not GHCNv4 peer article doing this, provenance is lacking. The notion that the ISTI peer article does this is wrong. ISTI web site specifically says it is not run through pairwise and that is a later step.

I concluded, thus K15 uses GHCN v4 alpha consistent with the file name. Russ Vose then said, ‘no it’s version 3’. However, then Jay Lawrimore said, ‘John’s right, it’s version 4’. There was some awkward silence and, since the meeting was already over time, folks just left. So, contrary to Thorne I do meet with these folks and I discussed this very issue with them AND Jay Lawrimore who actually runs the GHCN data said I was right. Thus, Thorne is wrong.

  1. [the SST increase] ‘was achieved by dubious means’

The fact that SST measurements from ships and buoys disagree with buoys cooler on average is well established in the literature. See IPCC AR5 WG1 Chapter 2 SST section for a selection of references by a range of groups all confirming this finding. ERSSTv4 is an anomaly product. What matters for an anomaly product is relative homogeneity of sources and not absolute precision. Whether the ships are matched to buoys or buoys matched to ships will not affect the trend. What will affect the trend is doing so (v4) or not (v3b). It would be perverse to know of a data issue and not correct for it in constructing a long-term climate data record.

Response:  The issue is correcting the buoys to ships, and the overall uncertainty of the data set and trend, in view of these large adjustments

  1. ‘They had good data from buoys. And they threw it out […]’ 

v4 actually makes preferential use of buoys over ships (they are weighted almost 7 times in favour) as documented in the ERSSTv4 paper. The assertion that buoy data were thrown away as made in the article is demonstrably incorrect.

Response:  Verbiage used by David Rose is not the key issue here. The issue is the substantial adjustment of the buoy temperatures to match the erroneous ship values, and neglect of data from the Argo buoys.

  1. ‘they had used a ‘highly experimental early run’ of a programme that tried to combine two previously seperate sets of records’ 

Karl et al used as the land basis the ISTI databank. This databank combined in excess of 50 unique underlying sources into an amalgamated set of holdings. The code used to perform the merge was publically available, the method published, and internally approved. This statement therefore is demonstrably false.

See response to #4.

What next?

What needs to happen next to clarify the issues raised by John Bates?

We can look forward to more revelations from John Bates, including documentation, plus more detailed responses to some of the issues raised above.

An evaluation of these claims needs to be made by the NOAA Inspector General. I’m not sure what the line of reporting is for the NOAA IG, and whether the new Undersecretary for NOAA will appoint a new IG.

Other independent organizations will also want to evaluate these claims, and NOAA should facilitate this by responding to FOIA requests.

The House Science Committee has an enduring interest in this topic and oversight responsibility.   NOAA should respond to the Committee’s request for documentation including emails. AGU and other organizations don’t like the idea of scientist emails being open to public scrutiny. Well, these are government employees and we are not talking about curiosity driven research here – at issue here is a dataset with major policy implications.

In other words, with the surface temperature data set we are in the realm of regulatory science, which has a very different playbook from academic, ‘normal’ science. While regulatory science is most often referred to in context of food and pharmaceutical sciences, it is also relevant to environmental regulations as well. The procedures developed by John Bates are absolutely essential for certifying these datasets, as well as their uncertainties, for a regulatory environment.

803 responses to “Response to critiques: Climate scientists versus climate data

  1. The articles at the Guardian and CarbonBrief about all this, could not even bring themselves to mention Dr John Bates name.. such is the state of journalism/fact checking these days.

    • BarryW could not even bring himself to link to Zeke’s piece or identify it’s topic. Zeke’s post is not about “all this” but about:

      In an article in today’s Mail on Sunday, David Rose makes the extraordinary claim that “world leaders were duped into investing billions over manipulated global warming data”, accusing the US National Oceanic and Atmospheric Administration (NOAA) of manipulating the data to show more warming in a 2015 study by Tom Karl and coauthors.

      What he fails to mention is that the new NOAA results have been validated by independent data from satellites, buoys and Argo floats and that many other independent groups, including Berkeley Earth and the UK’s Met Office Hadley Centre, get effectively the same results.

      https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise

      Does JohnB make himself that claim? I’m not sure he ever does. It’d be hard to do, considering the point of Zeke’s post.

      • hey – In Willard’s eyes – I’m en par with a major international newspaper, and a climate PR website CarbonBrief (totally funded by the big green lobbyist – The European Climate Foundation) that has burnt threw a couple of million Euros since it started existing. cool.. with respect to journalistic standards

        seriously Willard what the **** are you on.. the link to Zeke is in the blog post above..

        Why do you think Zeke can’t be bothered to mention Dr John Bates, a Principal scientist at NOAA (and former AGU board member) by name. Did he just forget, or disclosing his name, etc, might just makethe reader think a little.

      • BarryW’s mindprobing, peddling, and JAQing off doesn’t counter my point that Zeke’s article was about David Rose‘s claim, not JohnB‘s, contrary to BarryW’s “all about this” innuendo.

        Zeke’s post wasn’t “all about this,” BarryW. It was about the fact that the new NOAA results have been validated by independent data from satellites, buoys and Argo floats and that many other independent groups. This undermines the claim that data has been manipulated.

        This is why Judy’s dismissing David Rose’s verbiage as mere verbiage.

        This is why she insists on the process.

        Playing the INTEGRITY ™ card while washing one’s hands after handing out a story to a yellow journalist is a thing of beauty.

        Please, BarryW, do continue.

      • Willard – my criticism was of the journalistic standards of the Carbon Brief and the Guardian who are both make rebuttals of a specific David Rose article.
        That David Rose article leads with a photo of Dr John Bates, the article is about Dr John Bates as a NOAA whistleblower, that article mentions Dr John Bates,(a Principal Scientist at NOAA, and former AGU board member) TWENTY THREE times by name…

        yet both rebuttals at the Guardian and the CarbonBrief fail to mention John Bates by name at all.. And now according to Joshua (elsewhere), this is my “conspiracy theory” – LOL..

        nah. just s*** partisan ‘journalism’ . and of course at Ken Rice’s blog (AndTheresphysics commentator here) I cannot make a comment…
        https://andthentheresphysics.wordpress.com/2017/02/05/expose-david-rose-does-not-understand-baselines/#comment-91197

        what a ***** echo chamber it is that Ken runs.

      • BarryW – my criticism was about your poor choice of words, which led you to misidentify the topic of Zeke’s post. Once we get clear on that, you criticism of Zeke’s “journalistic standard” falters.

        It’s quite simple, really.

        The Auditor wrote enough “say my name” posts for anyone who has a modicum of experience in these matters to know what you’re dogwhistling with this.

        Now, since you do continue, let me help give you a hint to where we’re going:

        Our results, as you can see in the chart below, show that the new NOAA record agrees quite well with all of these, while the old NOAA record shows much less warming.

        https://www.carbonbrief.org/factcheck-mail-sundays-astonishing-evidence-global-temperature-rise

        Right in the middle of Zeke’s post we clearly see a reason why JohnB would lukewarmingly cling to the old data. If we believe what a new Denizen said on the previous thread regarding “NOAA policies governing ISI,” that would raise concerns about JohnB’s honest brokering of that whole masquarade.

        Shall we get his emails?

      • Barry – You won’t be surprised to hear this, I imagine?

        Hear, hear Willard!

      • “…accusing the US National Oceanic and Atmospheric Administration (NOAA) of manipulating the data…”
        And yet the simple expedient of following NOAA’s own policies would have undercut such an accusation right at the start – dare I suggest that this is the whole point of such policies!
        So you have a choice – follow policy and procedure and have an iron-clad guaranty that even should the policy later prove less than optimal, no-one can accuse you of being “tricky”. Or, ignore the policy and spend ten times as long trying to justify not doing so and even then put up with people claiming you “fudged”, “lied”, “cheated”, whatever.
        Bit of a no-brainer, ain’t it?
        Yet here we are…. go figure!

      • > And yet the simple expedient of following NOAA’s own policies would have undercut such an accusation right at the start

        This presumes this hasn’t been done, which is far from beying obvious.

        This also presumes audits sometimes end, which runs contrary to the common knowledge that audits never end.

        This finally presumes that David Rose’s accusation is somehow relevant to John Bates, something that runs against Judy’s “verbiage” dismissiveness.

        The argument is therefore invalid.

        ***

        > dare I suggest

        You’re on the Internet.

        Dare all you want.

        I’ll date to suggest that girls should wear longer skirts if they don’t want to get raped.

      • “This presumes this hasn’t been done, which is far from beying obvious.”

        Sure – I simply took the original post at face value. Given the lack of a response suggesting it WAS done, I can only conclude that either it wasn’t done, or it was done but no-one has bothered to point out that it was. Who knows? What I do know is that the response included some rather nit-picky stuff along with the meat, and I find it hard to believe that someone who did follow such procedure wouldn’t point out that they had.

        “This also presumes audits sometimes end, which runs contrary to the common knowledge that audits never end.”

        It’s not the end of the audit that matters, it’s the start – you can’t prevent someone starting an audit, but you can make sure there is no obvious reason (other than completeness) to start one.

        “This finally presumes that David Rose’s accusation is somehow relevant to John Bates, something that runs against Judy’s “verbiage” dismissiveness.

        The argument is therefore invalid.”
        Only if you accept Judy’s dismissal as definitive and final – not everyone can or will (and you will note that I make no comment regarding my own opinion on that score)

      • > What I do know is that the response included some rather nit-picky stuff

        I suppose this includes the fact that Zeke was refuting David Rose’s point, and not John Bates’.

        A point David Rose buried in his headline.

        A point that every single pingback at Judy reproduces.

        A point John Bates has yet to endorse.

        This is the point discussed in this sub-thread.

        It may not be the best sub-thread to deplore nitpicking.

        It may not be science, as teh Koonin once said, but it seems quite important to me.

      • “…common knowledge that audits never end”

        There is no such common knowledge. I have known of many audits that ended. Several of them I ordered to be done to my contractors. Audits are part of your quality assurance process. The point about a routine audit such as this isn’t that you expect malfeasance. It is to:
        1. Make sure the company (or other whoever is being audited) knows that there is somebody knowledgeable about the subject matter inspecting their work.
        2. Create a continuous improvement process, whereby each audit is used to improve the process. The next audit will check that the corrective action requests have in fact been done and created a real improvement in the processes used.

        Overall, a well run audit does wonders for the quality of work. NOAA’s failure to adopt proper quality assurance processes is extremely puzzling. Their approach has been made to look more political than scientific.

      • Mark,

        You must be new here. I was of course referring to a metaphor. As far as real audits are concerned, here are some points that may deserve “due diligence,” if you don’t mind me using a phrase the Auditor himself used quite a bit.

        First, the kind of “audit” relevant to climate stuff are more like what Auditor General does for governments than what accountants do for corporations. Corporations are private, while scientific institutions are not. The Auditor General represents the Public, i.e. the sum of private citizens, while an auditing firm represents private parties. I’m not sure who granted the Auditor that power to represent the Public, nor why His domain name ends with an .org, but such is life. Reality works in mysterious ways.

        Second, we should distinguish processes and products. The concept of an audit can refer to a yearly event, which leads to a report. It could also refer to the process of having one’s books checked each year. Think of how seasons end while the seasonal cycle does not. Or think of the difference between an R routine and a server.

        Putting those two together should be enough to understand why audits never end. The picture is not complete, of course – we still have to distinguish “calls” for audit like the ones we continuously hear in right-wing hit pieces like David Rose’s “verbiage,” which reinforces the never ending aspect of it. The picture only works by analogy, for it’s obvious that the calls for audit we’re witnessing work like proxy wars to cloak stealth advocacy.

        Hope this helps,

        W

      • “I’ll date to suggest that girls should wear longer skirts if they don’t want to get raped.”

        Classy Willard. However it does add insight to your fascination with pom poms and cheerleaders.

      • If Denizens don’t want to be reminded that some the excuses they profer to justify their online abuses have already been used in most violent contexts, they know what to do, TimG.

      • I’m curious. Why hasn’t there been a big stink about Spencer and Christy’s UAH tropospheric satellite dataset? It’s still beta version 6.x and has been used for almost 18months now. Their paper explaining the underlying changes has still not been published.

        Mear’ and Wentz from RSS updated their satellite data set to Version 4 last year. They waited until after their paper has been published to release the data set. The old Version 3 (which is closer to UAH 6.x beta) runs colder because of various issues outlined in their paper.

        Yet Judith Curry, Ted Cruz etc run around claiming “The satellite data is the best data we’ve got!” (referring to UAH). Despite the satellite data having much higher error range than land based temperature data.

      • Barry Woods being violent or abusive.

        Right.

      • “Contrarians” does not equal “each and every single contrarian,” TimG. Nor does it equal “each and every single contrarian each and every single time.” This thread should provide enough evidence for what I’m suggesting. You want an example?

        Even if BarryW never has been violent or abusive, what I’m saying could still be correct. His earlier gaslighting should be enough to disprove that, but I recall some of his old tweets, before he blocked me when I was retweeting his masterpieces. You want an example?

        In both cases, beware your wishes.

      • I wish the Orioles have the best rotation in the major leagues and it leads to a World Series title. Preferably more than one.

      • One rotation ought to be enough.

  2. Dr. Curry. What is needed now, is proof – not more accusations – of Tom Karl and NOAA’s Scientific misconduct. Your last post give none.

    • Rune, one does not need obvious misconduct to alter results over time. Just consistently keep applying pressure in one direction and the desired results are obtained. I believe Dr. Bates indicated that was Karl’s method.

      When I first read about adjusting ARGO float data to ships’, I was floored. The propagation of data (warming) error forward had to be significant as the percentage of ARGO data increased from about 10% to about 90%. Huang’s recent 0.07 degree C estimate seems low.

      • I don’t think Argo even was included in Huang et al(2015,) that where older types of floats. And since these floats had been used together with engineintake over a periode of several dekades, it was necessary to homogenize the data to get a consistent anomaly.

        When it comes to proof, Bates makes this statement:

        “A NOAA NCEI supervisor remarked how it was eye-opening to watch Karl work the co-authors, mostly subtly but sometimes not, pushing choices to emphasize warming. Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets—in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.”

        This is not made by a WUWT jerk, but by a former senior officer at NOAA. It’s equivalent to a character assassination of the scientific culture in NOAA, since both Karl and Peterson where authors in Karl et al. (2015).

        I think such claims need more hard evidence than what somebody felt, or some unspecified “pressure in one direction” behavior.

    • Steven Mosher

      Her Witness in Chief Just Recanted.

      The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

      • I think what will end up bothering the public are the adjustments made to buoy measuments in the direction of old salty ship measurements. It wont sit well. Well i guess merging the two datasets meant both had to come together…”or did they”? Is what people will be asking. This is all Lamar will need going forward. The real mess will come out when the emails are released. Expect a lot more crap to fly. This is just the beginning

      • For an English major it is surprising you don’t understand the meaning of the word Recanted Steven.

      • What would Bate’s think of Spencer and Christy’s UAH tropospheric satellite dataset? It’s still beta version 6.x and has been used for almost 18months now. Their paper explaining the underlying changes has still not been published.

        RSS updated their satellite data set to Version 4 last year. They waited until after their paper has been published to release the data set. The old Version 3 (which is closer to UAH 6.x beta) runs colder because of various issues outlined in their paper.

        Yet Judith Curry, Ted Cruz etc run around claiming “The satellite data is the best data we’ve got!” (referring to UAH). Despite the satellite data having much higher error range than land based temperature data.

    • >>I think what will end up bothering the public are the adjustments made to buoy measuments in the direction of old salty ship measurements.

      I think not. K15 is history, and even You don’t seem to understand that Karl was not a coauthor in the Huang paper (ERSSTv4), and where is the criticism of that? Most people, even here on this blog, seems to struggle with ships up and floats down and being unable to understand the simple logics.

  3. Trump’s opponents found that they could overturn his immigration ban through the courts. Expect similar challenges to his shakeup of the climate establishment too. This will be a fascinating battle between the executive and judicial branches of government.

    • It’s evident that Trump lacks the education, training, and vocation, to serve as a politician in a large structured burocracy. However, this is easily remedied by hiring people who are willing to do the job properly. And it’s clear Darth Banon isn’t the right person.

      However, an efficient manager can get parts of what Trump wants to accomplish done. For example, several months before he left office, Obama induced an inmigration service policy change, and we started to see Venezuelans who held business and tourist visas being taken to the Homeland Security detention centers in Florida. Obama simply didn’t want to see Venezuelans running away from the Maduro dictatorship seek political asylum in the USA, and he tightened the screw without any fuss whatsoever.

      Another case: just before he left office Obama ended the wet foot dry foot policy for Cubans, which led to Cubans being taken to the detention center upon arrival in Miami even though they held valid visas. Did the USA media raise a fuss? Did senators cry? Of course not. Those of us who flee communism from Cuba and Venezuela have a pretty weird situation: many right wingers despise us because we are inmigrants, and the left wing hates us because we fled communism and have a tendency to vote on the right. Obama made sure the paper work was done right, picked on a small group of people unable to elicit media sympathy, and got that job done.

      Trump can do exactly the same, all he has to do is get the right documents, make sure congress and big media are ready to back him. If he learns how to work effectively we should see changes. And I’m afraid many of those changes aren’t necessarily something many of us will agree with.

      • Fernando, your knee-jerk comment about Bannon exposes your ignorance. He just might be “perfect” for this moment in history.
        https://qz.com/898134/what-steve-bannon-really-wants/

      • I guess Darth Banon is perfect to create ill will towards the USA, incite Trump to have that blonde Kelly do informercials and make Trump look like a dummy?

      • Not evident at all.

        What is appearing to be evident is that Trump has no intention of conducting business as usual. He knows that is not what the people who voted for him want.

        Trump intends to shake things up and those who have prospered under the old order should be running scared. That order is likely to change. Fortunately the US is in a far better position to ride that change than much of the rest of the world.

      • Trump just got the best present he could have asked for. He did what he said he was going to do, what most of the people who voted for him WANTED him to do. And the left predictably went to the courts to stop him. The people can all now see that the Left is more interested in letting ‘Refuges’ and ‘Asylum-Seekers’ into the country (not to mention Left voting Illegal aliens) then in securing our borders, AS THE PEOPLE WANT.

        if the Left is lucky, all they will get is a decrease in voters in the next election. If they (and the rest of us) are unlucky, what we will get is some Terrorist who gets into the US because of this and manages to blow something up.

        I Guarantee if that happens, the people will know who to blame for letting them in.

      • I guess nobody explained to you how the USA political system works, the three branches of government, and why a president isn’t King? The courts will overturn Trump simply because his decree was sloppy. What in Mexico they call “una chapuceria”

      • Not sure Fernando whom you are referring to, but I understand how our government works. Am old enough to have had to take civics classes in public school. Was even selected for Boys State (District of Columbia). Apparently I played a significant role in our side winning a majority of the election contests, as I got the top spot of candidate for Governor. Learned my first lasting lesson on politics – the truth is not always your friend. Instead of making a speech about all of the made up policy positions I would take, I told everyone that I had worked hard to get to this position and would appreciate their vote because it would be really cool to be elected Governor. In hindsight that was not a productive decision.

      • cap6097,

        Thanks for the link. It made me go back to Edmund Burke’s ideas, whom I’d forgotten about. He expresses exactly what I feel.
        Yes I’d like us all to have Freedom, Human Rights etc, but one person’s freedom is another person’s restriction. What we do seem to have lost is old-fashioned responsibility. I may not ever want to meet some of the poorest in our country, UK, but if the UK doesn’t look after them, nobody else will. Somethng most of our immigration-loving politicians never think of.

    • Buoy oh buoy! “You expect similar challenges to his shakeup…” On the other hand you don’t know so this just talking at this point; he might go a better proper course of pulling back the curtains and making everything pragmatic, transparent and data driven… for a change.

  4. Judith

    I do not know if it was intentional or not but your use of the word verbiage has a slightly derogatory tone.

    http://www.thefreedictionary.com/

    I hope it is not a sign of a loosening of relationships with David as, despite what others might try to claim he is a first rate journalist, who, whilst not always one hundred percent correct, nevertheless constantly raises important issues

    Tonyb

    • I fear we’re going to have to agree to disagree on this one Tony! Here’s my considered opinion of Mr. Rose’s journalistic pedigree:

      http://GreatWhiteCon.info/tag/david-rose

      Surely not EVER one hundred percent correct is much nearer the mark?

      • Tony B, Jim Hunt is an expert on never being 100 % correct.
        Indeed he asked me to point out occasions on which he is wrong.
        A paradox really. With so much expertise in never being correct his statement is obviously non correct.

      • Mornin’ angech (UTC)

        Please provide a link to credible evidence of an occasion on which I was > 5% wrong.

        TIA

    • Steven Mosher

      I’d say she better back peddle from Rose and Bates or she might find herself, as a private citizen, on the wrong end of libel suit.

      Bates has recanted. I imagine in due Judith will have to come front and center and Clearly explain that She has no evidence of data tampering
      on Karl’s part. No evidence of a thumb on the scale, No evidence of ANY misconduct and ALL this case amounts to is Karl and company not going through Bates CDR program.

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

      • Bates can take a back seat now, the ball is rolling without left cover.

      • So “not disclosing what you find” is OK, then? Glad we’ve cleared that up.

      • Data tampering – apparently there are other terms you don’t quite grasp the meaning of.

        One can influence decision making in a specific direction without ever doing anything illegal or even unethical. Scientists make decisions on what data to include or exclude in their analysis all the time. Why would you think that process is immune to influence?

  5. I can confirm one of the points made by John Bates – that the GHCN adjustment algorithm is unstable. I’ve just done a blog post about this.

    It’s interesting to see Peter Thorne kind of acknowledge this (“a degree of flutter”). In the example I show, where I noticed it back in 2012, it’s 3 degrees of flutter. I don’t recall any climate scientists discussing or admitting this before. Peter O’Neill has also found this.

    • Paul,

      Speaking as a layman, that looks like a powerful confirmation to one of Bates’ claims. Two questions…

      (1) Have you or anyone else submitted this for publication anywhere, or sent it to NOAA?

      (2) Have you seen a response to these observations by anyone at NOAA?

      • there are a lot of scientist at NOAA tha are PAID to do just that.

      • Barry,

        “there are a lot of scientist at NOAA tha are PAID to do just that.”

        I don’t understand. Paid to do what?

      • No, it would be a lot of work to write something suitable for publication, you’d have to do a more thorough analysis of how it affects other stations.

        Both Paul Homewood and I did write to the GHCN people at NOAA and got a brief acknowledgement saying they would look into the problems but never anything more than that.

      • Paul,

        Thank you for the update. A disappointing but not unexpected response from NOAA staff. Perhaps now that Bates’ has brought these issues into the limelight, NOAA’s staff will respond to them. We can only hope.

      • Peter O’Neill has done a much more thorough analysis of many stations. There is indeed a degree of flutter. That is expected; the adjustment works by discriminating changes that it thinks are anomalous, and making adjustments if confirmed. Inevitably, some of these are line ball decisions, and reversed if neighboring data changes. This is not a software issue; it is the way the algorithm is intended to work. It removes bias, at the expense of increased noise (including flutter). The averaging process squelches noise, but not bias, so that is progress.

        Peter’s analysis showed fluctuations of mostly a fraction of a degree. Many stations were completely stable. I think this shows the limited propagation of the effect of fluctuating lineball decisions.

      • Here is the Ars Technica on the same topic, quoting Peterson to similar effect:
        “Bates also claims that there were bugs in the land station database software that were ignored in the Karl paper. But according to Peterson, the slight day-to-day variability seen in the software’s output was simply the result of the fact that new data was added every day. Stations that straddled statistical cut-offs might fall on one side of the dividing line today, and on the other side tomorrow. There was nothing wrong with the software, they realized. It was just silly to re-run it every single day.”

    • Steven Mosher

      Again, Paul is wrong.
      The adjustments dont change if you feed the algorithm the same data
      They DO change if you change the input data which is what Bates is misleading folks about.

      I challenged Mathews to download the code and do the simple test.

      He wont.

      He cant

      • Steven,

        “Paul is wrong. The adjustments dont change if you feed the algorithm the same data. They DO change if you change the input data which is what Bates is misleading folks about.”

        That’s exactly what Paul Matthews says. It is what he means by “unstable”: adding new data makes substantial changes to the historical record.

        “In other words, when new data was added to the system every week or so and the algorithm was re-run, the resulting past temperatures came out quite differently each time.”

      • …substantial…

        How “substantial”? Please quantify in order to support this assertion.

        …quite differently…

        How “different…”? Please quantify in order to support this assertion.

      • bernardj, it’s quantified in the blog post, and in my comment above. For the case of Alice Springs it is as much as 3 degrees C, though it’s less than this for most sites.

      • Single station daily variations get washed out in averaging over time and space

      • bernardj, it’s quantified in the blog post, and in my comment above. For the case of Alice Springs it is as much as 3 degrees C, though it’s less than this for most sites.

        What Eli said, with the added observation that you’re comparing apples to orange ideology.

        Did you even understand the point?

        What Eli said, with the added observation that you’re comparing apples to orange ideology.

        Did you even understand the point?

        What Eli said, with the added observation that you’re comparing apples to orange ideology.

        Did you even understand the point?

      • Bah, unresponsive submission…

        Still, it sort of works for the obdurate cognitive scotoma on display.

    • Steven Mosher

      Talk to Bates.

      The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  6. ‘Response: Well, in the new Huang et al. paper on ERSSTv5, it turns out that adjusting the ships to buoys results in a trend that is lower by 0.07oC. NOT exactly the same – in the climate trend game, a few hundredths of a degree actually matters.’

    By the way, where is the data archived for these results you’re publishing?

  7. I want to put this erroneous claim to rest.

    “other (sort of) independent analyses of surface temperatures”

    here is what is in the CCSP 1.1 report

    “The global surface air temperature data sets used in this report are to a large extent based on data readily exchanged internationally, e.g., through CLIMAT reports and the WMO publication Monthly Climatic Data for the World. Commercial and other considerations prevent a fuller exchange, though the United States may be better represented than many other areas. In this report, we present three global surface climate records, created from available data by NASA Goddard Institute for Space Studies [GISS], NOAA National Climatic Data Center [NCDC], and the cooperative project of the U.K. Hadley Centre and the Climate Research Unit [CRU]of the University of East Anglia (HadCRUT2v).”

    These three analyses are led by Tom Karl (NCDC), Jim Hansen (GISS) and Phil Jones (CRU).

    The differences between the three global surface temperatures that occur are a result of the analysis methodology as used by each of the three groups. They are not “completely independent”. This is further explained on page 48 of the CCSP report where it is written with respect to the surface temperature data (as well as the other temperature data sets) that

    “The data sets are distinguished from one another by differences in the details of their construction.”

    On page 50 it is written

    “Currently, there are three main groups creating global analyses of surface temperature (see Table 3.1), differing in the choice of available data that are utilized as well as the manner in which these data are synthesized.”

    and

    “Since the three chosen data sets utilize many of the same raw observations, there is a degree of interdependence.”

    The chapter then states on page 51 that

    “While there are fundamental differences in the methodology used to create the surface data sets, the differing techniques with the same data produce almost the same results (Vose et al., 2005a). The small differences in deductions about climate change derived from the surface data sets are likely to be due mostly to differences in construction methodology and global averaging procedures.”

    and thus, to no surprise, it is concluded that

    “Examination of the three global surface temperature anomaly time series (TS) from 1958 to the present shown in Figure 3.1 reveals that the three time series have a very high level of agreement.”

    Here is what Phil Jones told me in the paper [https://pielkeclimatesci.files.wordpress.com/2009/10/r-321.pdf]

    7. Degree of Independence of Land Surface Global Surface Temperature Analyses.

    “The raw surface temperature data from which all of the different global surface temperature trend analyses are derived are essentially the same. The best estimate that has been reported is that 90 – 95% of the raw data in each of the analyses is the same (P. Jones, personal communication,
    2003). That the analyses produce similar trends should therefore come as no surprise. Indeed, this overlapping of raw data between different analyses of multidecadal surface temperature trends is an issue which has not received adequate scrutiny with respect to the value added of more than one analysis.”

    It is time to reject this independence claim.

    Moreover BEST is also not independent as I summarized in my post

    https://pielkeclimatesci.wordpress.com/2011/11/01/comments-on-the-best-faq/

    https://pielkeclimatesci.wordpress.com/2011/04/04/sampling-bias-in-the-best-analysis-reported-by-richard-muller/

    https://pielkeclimatesci.wordpress.com/2011/10/21/comments-and-questions-on-the-best-analyses/

    Finally, here is what Zeke Hausfeather tweeted yesterday

    “@RogerAPielkeSr @ClimateOfGavin I agree that folks should strive to make it clearer that composite global temp records are not independent”

    In the above post from today, there is the statement

    “Zeke makes the point that CRUT and Berkeley Earth do not use the homogenized GHCN data.”

    True, but that is besides the point. Except for BEST they all draw from mostly the same raw data as the CCSP 1.1 report documents. Even BEST uses from mostly the same geographic areas. It is disingenuous to talk abut the processed data as, of course, they are worked on slightly differently.

    Roger Sr.

    • “True, but that is besides the point. Except for BEST they all draw from mostly the same raw data as the CCSP 1.1 report documents.”

      No, that is beside the point. There have been endless discussions of the reliability of the measured data, but that is not the topic here. Bates’ complaints are about processing, mainly adjustments. There are stories about the instability of GHCN adjustment etc.

      The fact that BEST and HADCRUT, even if working from the same raw data, get similar global averages gives confidence in the processing, including adjustment. All the things that are being talked about here.

    • Steven Mosher

      Roger

      1. None of your work on BEST establishes what you claim.
      2. Next it is out of date
      3. You never supply data or code and refuse to when requested.
      4. You have no CDR or process that insures the integrity of your
      approach.

      • All in bed together.
        One raw data set. 3 main fudged ,homogenised models.
        Sorry, WE don’t homogenise data, it’s those other people who give it to us.
        The hypocrisy of Stokes and M is breathtaking.
        On one hand they pretend to academic rigourousness, but when faced with it from ——- (no mention of name allowed) and Pielke they run around in circles disavowing standard scientific practice.
        Unbelievable. Totally shameful.
        And totally expected.
        Toss the honourable Zeke in that mix as well.

    • Well yes Roger. Newton was wrong – inductive reasoning based on data points is not true – it’s a paradigm. There is a cabal of insiders who systematically endorse some science and neglect into obscurity views that don’t quite gel. It was ever thus in science. Important science has been neglected for decades. Sir Gilbert Walker comes to mind. If they don’t neglect – they ridicule. And this gives permission to place 1000’s of scientists on global warming hit lists.

      There are two climate paradigms.

      1. Greenhouse gas warming at a 0.032W/m2 rate of warming and, originally, the absence of any natural variability. I was at ‘The Conversation’ yesterday where it was explained that they were writing software to model the Pacific Decadal Variation – and how the PDV presaged hyper-warming. They can’t model it – there are no numerical functions for it – and they have no clue as to the underlying causality.

      Here’s my idea on causality – https://watertechbyrie.com/2017/01/12/an-earnest-discovery-of-climate-causality/ – but going beyond hypothesis would seem a task of yet more decades.

      They are struggling to catch up to natural variability – and then to spin it.

      e.g. http://www.realclimate.org/index.php/archives/2009/07/warminginterrupted-much-ado-about-natural-variability/

      A compliant Swanson – and apparently warming is again not interrupted. Try to keep up.

      Our ‘interest is to understand – first the natural variability of climate – and then take it from there. So we were very excited when we realized a lot of changes in the past century from warmer to cooler and then back to warmer were all natural,’ The arch climate denier Tsonis

      2 Greenhouse gas warming at a 0.032W/m2 rate of warming set against a large background natural variability including – in the Holocene – climate extremes that put those of the 20th-century to shame.

      This time conflict over paradigms is vastly different. The cabal drops climate memes on compliant followers like crap from flying pigs. There are three sorts of followers.

      1. A political class who hype the meme to support ambitions of political and economic transformation.
      2. The collective of urban doofus hipsters who want to be smart, hip and superior to old white guys and sanitary engineers. They repeat the memes learned in doofus echo chambers with as little understanding as parrots
      3. The unsuspecting public who are told that 97% of scientist believe that greenhouse gases are causing warming at 0.032W/m2. Really – where are the other 3%? The implications being that it’s the only point of contention – obviously crazily unscientific – and that we should, therefore, dismantle global energy systems.

      The problem – or not – with the latter is that it is delusional. The ‘triumph’ of Paris saw the US commit to business as usual and to energy emissions rising 8.8% to 2030. What I hate most about it is the hype and the lies. We are left floundering in a post-truth world – a term that is appropriated as a marketing tool by Sci. Am. – with no clear path to rational policy and intellectual honesty.

      This is the rational policy being turned to by increasingly tuned in people in a world of hope,

      https://watertechbyrie.com/2017/02/06/the-climate-problem-and-the-solution/

      And this is what I have to say to doofus hipsters.

      https://www.facebook.com/Australian.Iriai/posts/1177863415663252

  8. This:

    “Well, in the new Huang et al. paper on ERSSTv5, it turns out that adjusting the ships to buoys results in a trend that is lower by 0.07oC. NOT exactly the same – in the climate trend game, a few hundredths of a degree actually matters.”

    Are you sure they are adjusting ships to buoys in ERSSTv5? I would not be so sure. For what i have read, the difference arises from different methods for areas without measurements. But of course: please give a link to documentation that ERSSTv5 adjusts ships to buoys.

    Besides this has been explained to many times. To get the offset between ships and buoys more correct, same result from adjusting the ships down or the buoys up.

    • “I think Dr Curry is however saying it would change the trend”
      But she is wrong, and Peterson is right. It is a matter of elementary arithmetic. Suppose you adjust by adding 0.12 to the buoy data only (all years). Then subtract 0.12 from everything. That have you got? The buoy data is back to where it was, and you have adjusted ship data down. The difference is just subtracting 0.12 from everything (all years). That can’t change any trends.

      • No Nick, I’m. Sorry but you are wrong. Fleet data traverses many stages of method and instrumentation; fleet data does not contain sufficient resolution to be fit for purpose. Were the raw numbers of a decent enough quality, the effect on the trend would not be a a massive question mark. As it stands, your claim that the trend is unaffected supposes you have access to resolution nobody else has. “Fit for purpose” is the term which is currently haunting Karl. But he knew this would be an issue. Dr. Curry is exactly right, that Karl ventured into highly dubious territory with K15 and now that the administration has changed he is unlikely to have left cover(which is why he has skated up till now).
        You may not understand this, Bates allegations are very serious and will be devoured by the HSC. And let’s be real for a minute; there is zero chance you would have authored K15 if given the chance.

      • Sophistry Nick.
        And possibly sloppy maths in this case?
        You are better at maths than I so I must be wrong but explain it again.
        The difference is not just subtracting 0.12 from everything (trend stays the same).
        You have added 0.12 to the buoy data, obviously the trend has to increase up by adding 0.12 to the total temperature load whilst the time period of ship/buoy data stays invariant.

      • angech,
        The point is that you have two datasets that measure the same thing. You know that there is an offset (i.e., when they overlap, the values are different). You therefore need to adjust these to be consistent. If you don’t, your estimate will be in error. It, however, doesn’t matter if you adjust one up, or the other down. Of course, making this adjustment will change the overall trend (relative to not making the adjustment) but having concluded that an adjustment is necessary, it doesn’t matter which way you make the adjustment.

      • Geoff Sherrington

        Nick,
        You seem not to be concerned that such an adjustment will change the estimated error. Error analysis, properly designed and performed, is central to detection of problems. It is not some optional add-on to satisfy journal face dressing needs.
        You comment that nothing changes is wrong. If nothing changes, why do it?
        Geoff

      • Geoff,
        “You comment that nothing changes is wrong. If nothing changes, why do it?”
        You are mixing up stuff. What I said was that varying between adjusting buoys up or ships down made a constant 0.12 difference to the whole dataset, and that can’t change trends. It also can’t change error estimates, which are based on variability about the mean. It just shifts the mean.

        The adjustments themselves also probably have little effect on statistical error. They relate to bias. It is observed that, when compared at close locations, there is an average 0.12°C difference between buoys and ships. It isn’t much, but you can’t ignore it. It is a bias and must be corrected.

      • Geoff,
        “You comment that nothing changes is wrong. If nothing changes, why do it?”
        I can expand on this with the algebra I set out here. It gives, for one month, the buoy-adjusted average as
        aB = A + wB*a/(wB+wS) = A + a*w*r/(w*r+1)
        where A is unadjusted average, r is the ratio of buoys:ships, and w is a weighting applied to buoys. The last term depends on known things – a, w and r, so there is no stochastic error component. That is all in A, and so is the same as for aB.

        Why do it? Because over time, r is changing. As it goes from 0 to large, the term added to A goes from 0 to a.

      • Reference this comment by …and Then There’s Physics | February 6, 2017 at 4:52 pm |

        I believe (but I’m not 100 % sure) there ought to be a slight difference in the final product if the two data sets (ships and buoys) have different distribution over the work space.

        I’ve found this issue when we were preparing reservoir descriptions. Say you have a map of the USA, sampling method A is uses interstate routes and can measure up to 20 miles on either side, sampling method B uses any road and measures up to 10 miles on either side. Neither method takes data in high crime areas. Figure out the implications if you are measuring the change over time of a given parameter.

      • fernandoleanme: there ought to be a slight difference in the final product if the two data sets (ships and buoys) have different distribution over the work space.

        That is an excellent point.

        A larger question whether they do in fact make exactly the same adjustment to every data point, when the ship measurements are not homogeneous in either time or space.

        Nick Stokes: It is a matter of elementary arithmetic. Suppose you adjust by adding 0.12 to the buoy data only (all years). Then subtract 0.12 from everything.

        Is that in fact an accurate description of what they did and what happened during the time of only ship-based measurements? Would that make sense if the ship-based measurement bias was not constant? Was that measurement bias constant?

      • matthewrmarler,
        “Is that in fact an accurate description of what they did and what happened during the time of only ship-based measurements?”
        They add 0.12°C to the buoy readings. The rest is hypothetical to show that it wouldn’t matter numerically if they subtracted 0.12°C from ships. But yes, the reason why they probably don’t do the latter is that for most of the history, there weren’t buoys, so it may seem odd to adjust ships to them. But it would make no practical difference if they did.

        “Would that make sense if the ship-based measurement bias was not constant? Was that measurement bias constant?”
        We only have buoy-ship comparison data for the last two decades. In that time, a 0.12 discrepancy is observed. Assuming constant isn’t ideal, but it’s better than assuming zero.

        FL,
        “I believe (but I’m not 100 % sure) there ought to be a slight difference in the final product if the two data sets (ships and buoys) have different distribution over the work space.”
        Kennedy 2011 divided up into different oceans, and found small differences, on the fringe of statistical significance. I think ERSST uses local EOFs to minimise location uncertainty.

      • Nick Stokes: They add 0.12°C to the buoy readings.

        That’s at the right-hand end, most recent two decades, of the temperature series, isn’t that right?

      • “That’s at the right-hand end, most recent two decades, of the temperature series, isn’t that right?”
        Well, it’s to all the buoy readings. But yes, most are in that timeframe. That is probably why they choose to add to buoys rather than subtract from ships, which would have identical effect. Less years to change.

      • Nick Stokes: That is probably why they choose to add to buoys rather than subtract from ships,

        I don’t think any reasonable adjustment is possible, because the ocean temps prior to the buoys are such a heterogeneous lot: geographically biased, diverse measurement types, imbalance of type and timing (i.e. lots more combat fleet activity during WWII.) So, I am aiming to some kind of reasonable evaluation of information that has been hyped.

        Perhaps a random draw of adjustments, so show how uncertainty in the “adjustment process” leads to increased uncertainty in the results.

        The “pause” hardly mattered, except that it was totally unpredicted by everyone warning us about imminent danger, and then denied after it was underway, then the topic of numerous explanations (even as some people kept denying it). Then the Karl et al “pause-buster” paper was hyped, though it had flaws (even if you accept the main result). Now the fact that Karl et al deviated from written SOPs shows (to most) the political bias (since it was published just in time for Obama to use in Paris). Was there in fact another justification for deviating from the written standard?

      • “I don’t think any reasonable adjustment is possible”
        So should the whole history be abandoned? The “pause”, after all, was based on that. There is evidence that buoys read 0.12°C cooler than ships. So why is adjusting by 0.0 better than adjusting by 0.12?

        “Now the fact that Karl et al deviated from written SOPs shows”
        It isn’t a fact. It is an allegation by John Bates, who has proved unreliable on several other matters.

      • Nick Stokes: So should the whole history be abandoned?

        I like my idea of a random draw of adjustments, so see how uncertainty in the adjustment affects uncertainty in the estimated trend.

        The idea of the “pause” was not based exclusively on ocean temps.

        Going forward, I expect we’ll give less and less weight to the earlier ocean temps, due to their oft-reviewed limitations.

        One approach is to treat the date of onset of the buoy record as a knot in a piecewise polynomial spline, and fit the early ocean data and buoy data subject to the usual constraints at knots: equality of the model and its first two derivatives at the knot. Because of the complexity of the shape of the record, a few knots might be needed at earlier times, say at 25-35 year intervals.

      • MRM: equality of the model and its first two derivatives at the knot.

        I mean, with and without those equalities at the knots that confirm smoothness on the fitted model, to see what emerges as the best adjustment of the levels at the knot, and the effects on the slope estimates of the equalities.

    • Owen,
      “As it stands, your claim that the trend is unaffected supposes you have access to resolution nobody else has.”
      You seem to have this fixation on ship data quality invalidating K15. You don’t deal with the point that K15 is not introducing ship data, which has been used for many years. It is about how to properly merge buoy and existing ship data.

      But in any case, my claim about trend is a matter of arithmetic. The arithmetic is the same for good data and bad.

      • To be pedantic (but I think this needs to be spelled out), K15 is not about how to properly merge buoy and ship data. That was covered by Huang et al. 2015, and previous papers looking at ship/buoy biases.

      • Nick, yes now u are beginning to understand. Fleet data has been used for years, but for what has it been used? And is it appropriate to use it now? The math you are using, I understand it, deals with two sets of numbers and the difference between them. Yes. The deeper issue has always been Karl’s choice in rolling junk numbers into ARGO numbers. Why has he done this if not in an attempt to subtly erase the pause? It’s in the @! $×ING title. So now what happens? NOAA is going to see some changes. Big changes.

      • “The arithmetic is the same for good data and bad.”
        Only true if both datasets are given the same weighting – which they are not – and/or the proportion of each stays constant, which they don’t.

      • Owen,
        “Karl’s choice in rolling junk numbers into ARGO numbers”
        I think you need some catch-up on this whole discussion. These aren’t ARGO numbers. But in any case, the fact is that we have a long history of ship data, carefully assembled. We have a short history of data from surface buoys. Karl is writing about issues in adjusting one to the other. The ship data isn’t going to be ditched, even if Karl were to wish so. It has to be got right.

      • No, ship measurements are not carefully assembled. Please provide evidence of a full survey. If you cant do that much please concede the point and move on.

    • I’d like to just set out the simple arithmetic of adjustment, since it seems to be causing trouble to some. Suppose you first average ships and buoys separately over a period of time, say a month. Whatever you do to weight or adjust, you do equally to members of each class, so you do the same to the average.

      In that month, there are nS ships, average S, and nB buoys, average B. The combined average A will be a weighted sum:
      A = (wB*B + wS*S)/(wB+wS)
      If you want an unweighted average, then wB=nB and wS=nS. But if you want to upweight buoys by 7, then wB = 7*nB.

      Now suppose you adjust buoys up by a (0.12). Then
      Ab = (wB*B + wB*a + wS*S)/(wB+wS) = A + wB*a/(wB+wS)
      Or if you downweight ships by a, then
      As = (wB*B + wS*S – wS*a)/(wB+wS) = A – wS*a/(wB+wS)
      Then
      Ab – As = a*(wB+wS)/(wB+wS) = a

      Constant a for each month, however A and B change. So no diffrence in trend or shape. This is true whether you upweight buoys or not.

      • Nick,

        I may be missing something, but I get a different answer if I assume the simplest weighting process, namely that the average temperature at any time is given by a weighted average proportional to the number of measurements by each source i.e.

        Tavg = (ns*Ts + nb*Tb)/(ns + nb)

        where Ts is the ship temperature and ns the number of ships, Tb is the buoy temperature and nb the number of buoys. If we then assume that the buoy temperature is higher that the ship temperature by a value delT, so that Tb = Ts + delT, then by substitution Tavg = (ns*Ts + nb*(Ts + delT))/(ns + nb). Simplifying, Tavg = Ts + nb*delT/(ns +nb). Obviously, as nb increases the average temperature increases in a smooth progression, with a warming “error” slope proportional to nb*delT/(ns + nb): when there are no buoys the ships totally determine the temperature and when there are no ships the buoys determine the temperature with a positive error slope in between. Can you be sure that something like this wasn’t done?

      • skjackso,
        ” Obviously, as nb increases the average temperature increases in a smooth progression, with a warming “error” slope proportional to nb*delT/(ns + nb):”
        Yes. I said the same with slightly different notation here. If you don’t correct, the temperature gradually slides from ship temp to buoy temp, which is 0.12°C cooler. That isn’t because of a climate change, but just the way of measuring. The correction counters that.

        In your math Tavg = (ns*Ts + nb*Tb)/(ns + nb) is Ts if nb=0, and Tb if ns=0.
        But with the correction as you put it, it is Tb+delT = Tb when ns=0 – ie, no change due to shift in measurement devices from ns=0 to nb=0.

      • Nick,

        That clears it up. Thanks for your reply.

      • “Constant a for each month, however A and B change.
        So no diffrence in trend or shape.
        This is true whether you upweight buoys or not.”

        So…. if it doesn’t make any difference, why do they even bother to do all this gimmics? That’s the vexata quaestio…

        Nice try, though.

      • ” if it doesn’t make any difference”
        It doesn’t make a difference whether you adjust buoys to ships or ships to buoys. The adjustment itself makes a difference – that’s what people are complaining about. But it has to be done. There is a known discrepancy between buoys and ships.

      • Geoff Sherrington

        Nick,
        Procedurally wrong. Anyone can take a pencil to write some algebra. The algebra might mean nothing because ivdoes not include adequate observation to account for all possible cause. Properly, youb would need to develop transfer functions from each ship each related buoy possibly for each weather season then add in variables like lat, long, rainfall that day, depth sampled etc. Your 2 lines of algebra are really avrsther largevmatrix, many of whose cells cannot be filled because data are not recorded.
        Further, I suggest you look hard at Kenneth Fritsch’s posts nearby to get a result more sophisticated than the linear least squares simple approach.
        Geoff

  9. This is a critique of David Rose &| the Mail on Sunday. The “anomalous baseline” graph displayed in Mr. Rose’s recent article in the Mail on Sunday is still “misleading” as we speak.

    Forgive my broken record impersonation, but is there any comment yet from Messrs Bates &| Curry on that teensy weensy little detail?

  10. David L. Hagen

    This may require action by the Inspector General of the Department of Commerce to override the deep political bias and vested interests at NOAA.
    Report Fraud, Waste, Abuse, & Whistleblower Reprisal
    But this should be done by principles involved John Bates (and Judith Curry?) with others joining in for maximum concentrated effort and impact. Note that the NOAA guidelines on peer review directly hinder the scientific process by preventing objections to models.
    That is directly contrary to Richard Feynman’s high standard for scientific integrity of examining every possible objection to a model. See Cargo Cult Science, 1974.

    • Steven Mosher

      Except bates Recanted now

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  11. And this:

    “Response: The issue is correcting the buoys to ships, and the overall uncertainty of the data set and trend, in view of these large adjustments”

    Judith seems to have missed that the results from comparing bouys only, Argo, ATSR sats to ERSSTv4 validates ERSSTv4.

    How is that possible?

    • Geoff Sherrington

      Ehak,
      It is possible because data completely unfit for purpose is being used in a web of very poor climate ‘science’ by people who display little understanding of sampling theory.
      They wrongly assume that an academic/mathematic analysis can correct wrong results. In real, practical life one does meet data sets that cannot be resurrected.
      Ship intake temperatures are clearly such a case. Strip out the emotion, discard the wish to have some data to support an invented story and be objective about the ship data.
      It is not good enough for its present use, is it? Not marginal, but clearly so full of irreversible problems that it should never have been floated from the start.
      People need to get some guts and go public with this magnificent example of the Emperor’s clothing.

  12. And of course this:

    “Response: Verbiage used by David Rose is not the key issue here. The issue is the substantial adjustment of the buoy temperatures to match the erroneous ship values, and neglect of data from the Argo buoys.”

    Judith is just repeating old misunderstandings. Try it yourself Judith. Adjust ships down and see if that changes the trend.

    Why don’t you?

    • OK ehak – for a layman – what is your point exactly ?

      Is it just that it doesn’t matter if you just use the buoy data (which is slightly colder) its the trend in the graph over time that matters, and that that would show the same trend ?

      I think Dr Curry is however saying it would change the trend :

      “Response: Well, in the new Huang et al. paper on ERSSTv5, it turns out that adjusting the ships to buoys results in a trend that is lower by 0.07oC. NOT exactly the same – in the climate trend game, a few hundredths of a degree actually matters.”

      Surely that is the issue then ?

    • “I think Dr Curry is however saying it would change the trend”
      But she is wrong, and Peterson is right. It is a matter of elementary arithmetic. Suppose you adjust by adding 0.12 to the buoy data only (all years). Then subtract 0.12 from everything. That have you got? The buoy data is back to where it was, and you have adjusted ship data down. The difference is just subtracting 0.12 from everything (all years). That can’t change any trends.

      • Nick, the new ERSSTv5 paper finds a 0.07C difference. Its not published yet and not generally available, but they did the work and found it made a difference

      • Surely “trend that is lower by 0.07oC.” means the trend over time is lower (ie not so marked ?) not the temperature values are lower ? Not sure how that would be though.

        Otherwise we are indeed talking about something trivial.

        Again I am very much an observer layman here.

      • “Suppose you adjust by adding 0.12 to the buoy data only (all years). Then subtract 0.12 from everything. ” What does that mean? Are the number of data points between the two sets of data exactly the same? If not, it seems it’s not a matter of simple arithmetic. Could you explain?

      • Judy, the change in ship-buoy differences in ERSSTv5 is not due to the choice to adjust the ships down to the buoys (vs. buoys up to the ships). Rather, its due to the use of a dynamic buoy-ship difference calculated for each month by comparing coincident ship and buoy measurements. This change reduces the average ship-buoy difference from the 0.12 C assumed by ERSSTv4 to 0.08 C. How this is applied (ships down by 0.08 C or buoys up by 0.08 C) would have zero effect on the resulting anomalies.

      • Thanks, Zeke

      • Zeke,

        Makes sense. Is the bias uncertainty not quite a bit bigger though?

      • Daniel Hofford,
        “Are the number of data points between the two sets of data exactly the same? If not, it seems it’s not a matter of simple arithmetic. Could you explain?”
        It still is the same. I have set it out algebraically here.

      • @Zeke Hausfather:
        “How this is applied (ships down by 0.08 C or buoys up by 0.08 C) would have zero effect on the resulting anomalies.”

        The only reason anomalies are so important is because the parameter of interest is the temperature trend and not absolute temperature. So why is the effort placed on lining up absolute temperatures? If there is buoy data and ship data overlapping in time, wouldn’t it make more sense to use the buoy temperature trend to calibrate the ship temperature trend?

    • Ehak, you’re putting your eggs in one basket; the trend stays relatively the same either way. Correct, (supposing the fleet data has been possessed to be fit for purpose…sufficient validation and resolution) and there you have it wrong. It not possible to achieve a proper fleet dataset without a full survey complete with engineering reports on each vessel relative to citing and more. In fact, the numbers once properly adjusted may look nothing like those used by Karl. So trend line is in question, do you follow? Procedural changes as well as instrumentation and materials make a full survey next to imposible; and so we can say the data is not fit for purpose. Fit for certain things, but not for a highly influencing paper, paid for by tax payers, meant to sway policymakers. Try to get all of this. And don’t be too surprised that’s heads are rolling as they should be.

      • Rt. Rear Admiral David Titley, former NOAA chief operating officer:

        In summary, the Mail on Sunday has found a disgruntled ex-NOAA employee and is using him to construct alternative facts about the climate. Unfortunately for all of us, the air will keep warming, the seas will keep rising, and the ice will keep melting, regardless of the Daily Mail’s fanciful claims and accusations. The real atmosphere is impervious to alternative facts.

        There is both a NOAA internal process on scientific integrity (my office ran it when I was at NOAA) and the opportunity to submit allegations of wrongdoing to the Department of Commerce Inspector General who, if there is reasonable evidence to substantiate the allegation, would undertake an independent investigation.

        Is the science bad? No. Karl et. al. was published in a high-prestige, peer-reviewed journal, where the reviewers were almost certainly eminent scientists external to NCEI and NOAA. More importantly, the Berkley group (originally founded to disprove the NOAA and NASA temperature recorded but ended up confirming their validity) as well as other external organizations, such as the UK Met Office, have all subsequently INDEPENDENTLY replicated the Karl et. al. results. That’s the gold standard of science, not some NOAA internal bureaucratic procedure that may or may not have been followed.

        Is the earth continuing to warm? Yes

        While the arguments about ARGO floats vs. WWII shipping sea water injection temperatures vs. Satellite SST’s are fascinating to the dozens of scientists who care about such things, our knowledge of global warming and climate change is built on overwhelming, independent lines of observational evident, understanding of the basic physics that goes back to the mid 19th century, and our ability to accurately project the overall global warming in computer models for over 40 years. Nothing in the Daily Mail article refutes any of this evidence.

      • JCH, your “… our ability to accurately project the overall global warming in computer models for over 40 years.” could indicate that you are an ideologue, bent on confusing the ignorant.

        Gees-us!

      • richardswarthout

        JCH

        Titley was at NOAA for 12 months. Up to now it was an innocent 12 months. He is now at Penn State, making things up to please the master.

        Richard

      • Just absolute nonsense.

      • JCH, which part is nonsense?

      • “Nature Publications” is no longer high prestige. They were purchased and are controlled by a small group of people I call “Baader Meinhoff II gang”.

        Given the political nature of “climate science” being practiced in the USA, all work financed by government grants and/or carried out by government employees needs to be published in a fully independent set of journals. The editorial board needs to be nominated by a fully bipartisan congressional committee with a 67% vote, and approved by the Supreme Court.

  13. Maybe Dr. Bates should have gone to the WaPo first.
    Anyway, the House and the Senate committees that oversee the NOAA will hear him out. The new NOAA management are not like the Obama stonewallers and will not hide emails from the committees. There is a new Sheriff in town. Get used to.

  14. I have never worked for a company that considered emails written by me on their email server, mine. Even emails sent to me by my family belonged to the company. If you are using a company or government email server, the email belongs to the company or government, period. Get used to it.

    • Have you ever worked for a company that had balance of powers between three branches?

      They have likely had all the relevant emails/documents for more than a year, and that means there is nothing in them.

      Clutch… clutch… clutch… It’s hilarious.

      • If the e mails were clear, mine would be, they would be released.
        No privacy concerns clear my name.
        They refuse to release them. There is dirt there.
        They will be forced to release them now.
        Whistleblower evidence of malfeasance. So it will all come out . 8 months max.
        Have some slow popcorn JCH.

      • It is common when a congressional committee asks for documents for the executive branch to initially refuse. They begin a song and dance. Sometimes the documents come out; sometimes they don’t.

        It not like a company.

      • Why is it that JCH, Nick and others are involving themselves in what is an interoffice feud, at best?

        Whatever truth (?) there is will eventually ooze out.

      • President Trump may not want to weaken the presidency.

      • President Trump is an 800 lb gorilla trying to overeat and use steroids to put on another 400 lbs. The rise of the imperial presidency is a fact, and if it doesn’t stop and get reversed, the USA is doomed.

      • More nonsense from JCH.

        1) what does balance of powers have to do with access and ownership of emails on government servers? – nothing

        And your claim that Smith’s committee have had all of the relevant emails for more than a year just shows you didn’t read the entire article you linked to. Otherwise you would have seen this:

        “Now NOAA has handed over the first tranche of emails to Smith’s staff, except for emails from agency scientists responsible for June’s study — Republican staffers opted to get non-scientist emails from NOAA.”

  15. Pingback: A whistleblower challenges NOAA’s climate data | Fabius Maximus website

  16. There is a simple test for all those criticizing Dr. Bates:

    Have you reproduced the results of K15 independently? If so, please publish the data and code.

    If not, then Dr. Bates’ point is made.

    • This is a nonsense point, repeated too often. Reproducible doesn’t mean you can reproduce it in a few minutes. K15 is the work of many scientists over many months.

      • —–s said it was now not reproducible,Nick. The data was there, the code was never available to Science or the general scientific and public audience. Tell us where the usable code and data is Nick, you usually have the resources.
        “Since the version GHCNM3.X never went through any ORR, the resulting dataset was also never archived, and it is virtually impossible to replicate the result in K15.”

    • I’d be happy to replicate K15 independently. Here is a replication of their ocean temperature component, including validation against two completely independent datasets: http://advances.sciencemag.org/content/3/1/e1601207

      Here is a replication of their land component using the latest GHCNv4: https://www.carbonbrief.org/wp-content/uploads/2017/02/NOAA-land-comparison-1024×819.png

    • And these, Zeke, are the data and code used by K15?

    • Steven Mosher

      Except bates Recants.

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

      • So, Mr. Mosher … did Bates “recant”? I’m not sure I’ve seen it enough in this comment thread. ;) Why don’t you repeat it 10^99 times so we can read it over and over again?

  17. ==} They are basically arguing that the NOAA surface temperature data sets are ‘close enough’ to other (sort of) independent analyses of surface temperatures. Well, this is sort of beside the main point that is being made by Bates and Rose,… {==

    Really?

    Looking past your description of Bates’ “main point,” let’s look at your defense of Rose that you offer here (while also explaining that Rose’s “verbiage” isn’t particularly important)

    The Mail on Sunday can reveal a landmark paper exaggerated global warming.

    Did the landmark paper exaggerate global warming if other (sort of) independent analyses show similar results?

    You parse Rose’s “main point” without noting that he prominantly focuses on “exaggeration of warming,” and without noting the conspicuous absence in his article of comment on other analyses, and you bypass his clearly inaccurate representation of other analyses…. and then claim that his verbiage “is not the key issue.”

    On another note, it would be interesting to see you address another criticism that you didn’t address:

    you point to this:

    –snip–
    ‘They had good data from buoys. And they threw it out […]’
    –snip–

    And respond by saying

    ==> Verbiage used by David Rose is not the key issue here. ==>

    Except that was a quote of Bates by Rose not Rose’s “verbiage.”

    Please address Nick Stokes’ responses to whether Bates’ statement (and Rose’s quotation).

  18. Prof Curry, thank you for this essay, and for posting John Bates’ essay.

    • Steven Mosher

      To Bad she Never Asked him the Probing question…
      Are you making an accusation against karl and his co authors?

      She never asked the Eight question.

      Instead, she posted innuendo and That Allowed Hacks Like Rose and Watts and Others to Turn a molehill into accusations of fraud

      The AP asked the right questions

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

      • Zeke Hausfather (2010):
        “a close examination of the residuals when satellite records are subtracted from surface station records shows no significant divergence over time compared to either UAH or RSS”

        What happened then?

    • mosher: She never asked the Eight question.

      It was good of you to comment.

  19. Questions to raise: how is ship’s data obtained? How is the buoy data obtained?

    Assuming the ship data is from manual reading of liquid-in-glass thermometers, are these thermometers suitably calibrated, scientific thermometers, and where are these thermometers located?

    Assuming the buoy data is from automatic electronic thermometers, how accurate are these thermometers, and how is the continuing accuracy of these thermometers monitored?

    • Ship’s data is acquired from the engine room cooling inlet temperature gauges by the engineers at their convenience.

      There is no standard for either the location of the inlets with regare especially to depth below the surface, the position in the pipework of the measuring instruments or the time of day the reading is taken.

      The instruments themselves are of industrial quality, their limit of error in °C
      per DIN EN 13190 is ±2 deg C. or ±4 deg. C for a class 2 instrument, as can be seen here: DS_IN0007_GB_1334.pdf . After installation is exceptionally unlikely that they are ever checked for calibration.

      It is not clear how such readings can be compared with the readings from buoy instruments specified to a limit of error of tenths or even hundreds of a degree C. or why they are considered to have any value whatsoever for the purposes to which they are put.

      • EXACTLY!!!

        End of friggin’ discussion.

      • I would just ignore all these thermometers and watch how ocean fish are migrating farther and farther from the equator. It’s how the biosphere reacts to warmer oceans that really matter. Fish and algae can’t read thermometers so they just move or go extinct.

      • Wow. What’s the point of combining ship data with buoy data if the ship data has a nominal uncertainty of ±4 deg. C and a real uncertainty of something more like ±8 deg. C when other issues are taken into account, i.e. the temperature gauges not being calibrated?

        I would think that in virtually every instance the buoy data by itself would be of greater value than buoy + ship.

        If it could be proven that the errors in ship temperature gauge data were symmetrical, that is equally likely to be up or down from the real, then I could see the possible virtue in combining. Maybe.

        Otherwise, you plot ship data when ship data is the only data you have. As soon as you have a significant number of buoys, you plot only buoy data. The transition between ship and buoy data should always be clearly marked, should not be smoothed, and probably should be discontinuous to make it clear to the reader’s eye that there is an abrupt change in what is being measured.

      • Jack Smith,

        Problem is fisheries scientists don’t know much about ocean fish populations and migration habits. Which may explain why the studies I’ve seen on this topic are all model based. They take model output from the GCM’s to give them expected changes in temperatures and input them into other models for fish migration patterns and come up with such incredibly good conclusions like “fish populations are doomed” based on the models saying the fish can’t swim fast enough to escape warming waters.

      • “Wow. What’s the point of combining ship data with buoy data if the ship data has a nominal uncertainty of ±4 deg. C and a real uncertainty of something more like ±8 deg. C when other issues are taken into account, i.e. the temperature gauges not being calibrated?”

        Because high-reading ones will be cancelled by low-reading ones most likely (given a random distribution of instruments).

        “In other words, accuracy describes the difference between the measurement and the part’s actual value, while precision describes the variation you see when you measure the same part repeatedly with the same device.”

      • Because high-reading ones will be cancelled by low-reading ones most likely (given a random distribution of instruments).

        I presume you can prove this with multiple peer reviewed papers?

      • “Because high-reading ones will be cancelled by low-reading ones most likely (given a random distribution of instruments).”

        Oh yes, very scientific. More than adequate to produce data trends to three decimal places on which trillions of dollars can be decided I’m sure. /sarc

        “In other words, accuracy describes the difference between the measurement and the part’s actual value, while precision describes the variation you see when you measure the same part repeatedly with the same device.”

        Which relates to the subject at hand how, precisely?

        In any case, when I need tuition from the likes of you on the difference between acccuracy and precision – something I have around half a century of personal experience dealing with in mission critical applications, I’ll ask you for it.

        Shouldn’t you be at work predicting a ‘Barbecue Summer’ or something?

      • cat,

        The analogy I like to describe the difference between accuracy and precision is the infantry one.

        Ask an infantryman the difference and he will tell you accuracy is putting ordinance on target with a circular error that includes the target.

        Precision is putting it within the smallest circle of error from your aim point. They are not always the same.

  20. “In other words, with the surface temperature data set we are in the realm of regulatory science, which has a very different playbook from academic, ‘normal’ science.”

    But the publication of Karl et al 15 should fall in the realm of academic, ‘normal’ science. Publishing a research article is not a regulatory action, right?

    • The mission of NOAA NCEI is operations, not research

      • It seems to me they are doing research too. And I don’t think it is a bad thing that their methods are published in scientific journals.

      • Their paper’s title is a scientific one.
        “Possible artifacts of data biases in the recent global surface warming hiatus”
        They are not releasing a new official dataset, but pointing out severe problems with the old ones. A valid subject for a paper.

      • Precisely, Jim. The important contribution of their paper is the method. It should be applicable also to newer versions of the data series they use, like the forthcoming ERSSTv5.

      • Scientific operations frequently produce scientific research.

        It happens in a lot of places. I’m surprised that you don’t know this.

  21. I found this related post, presumably coroborationg one of Bates’ observations:

    https://cliscep.com/2017/02/06/instability-of-ghcn-adjustment-algorithm/

    (The instability of GHCN-adjustments)

    • Paul Mathews linked it earlier. Shows that Bates was correctmon his land imstability point.

      • “Shows that Bates was correct…”
        Hang on there. We need a bit more data on whether Paul Matthews is correct. If there is such instability, I think we would have heard a lot more about it. It doesn’t show up in the indices. People complain about their frequent changes, but they are very small.

      • Nick,

        “We need a bit more data on whether Paul Matthews is correct.”

        Which is why a response by NOAA staff to Matthews’ email would have been nice. They said they would look into it.

        “If there is such instability, I think we would have heard a lot more about it.”

        That’s an aggressive assumption. History, modern and past, overflow wit examples of technical details inconvenient for the institution that are overlooked — until they emerge to bite. Even when the stakes are high, group-think often overrules evidence.

      • Steven Mosher

        Run the code.
        Its been available for years.

        Not unstable

    • This is disturbing stuff about the structural instability of NOAA’s adjustment algorithm. Even more disturbing is that when shown evidence of the issue, NOAA apparently said nothing in response.f

      Nick, If it doesn’t show up in the averaged indices that might why it was not corrected early on. It’s a little like saying that the Lift values are pretty close, so the fact that the pressure distributions are very different is irrelevant. Not a good answer in my opinion.

      • DPY,
        “This is disturbing stuff about the structural instability of NOAA’s adjustment algorithm”
        Maybe, but we need better evidence (eg posting the deviant datasets). The PHA code has been publicly available for years. There is a list of changes here. No mention of anything like that. I think a lot of people would have noticed such instability.

      • I wish the people complaining would put numbers and algorithms to the complaints so we could quantify the errors (and fix them). It might also turn out not to make a material difference. Are people interested in getting the best science or just slowing down science with complaints and making mountains out of every molehill they see?

      • Jose, private citizens do not have the resources to do large detailed studies.

        The issue is that universities, governmental agencies and public employees are not forthcoming. Governmental entities tend to suppress inconvenient facts. Particularly egregious are non-response to information requests and denial or suppression of problematic data and processes.

        Given the long history of worldwide political and bureaucratic wrongdoing, citizens are rightly concerned about freedom of information and access to governmental scientific processes. Non-responses and outright stonewalling will make many suspicious of motivations, political or otherwise.

        Just one small example of suspicious activities will have to suffice: Our government’s huge song and dance about 2016 being the hottest on record.

        The problem? No mention of satellite data.

        It appears it is a political exercise, not an exercise in informing the public.

      • None of this discussion makes a material difference to the temperature record. K15 didn’t make a material difference to the temperature record. This is about process, and was politics influencing process?

      • “The problem? No mention of satellite data.”
        There was plenty of mention of satellite data. The problem? It was also record warm.

      • Wrong-o, Nick-o.

        The scary presentations were based on surface data.

      • Steve McIntyre has checked and confirmed what I said about the instability of the GHCN algorithm. The temperature at Alice Springs in 1880, as reported by GHCN at different times during 2012, varied by over 4 degrees C!

        https://twitter.com/ClimateAudit/status/829071635852443648

      • That level of instability is unbelievable. This may be the most significant part of this post. 4 degrees C!

        What does Nick S say about that finding? Very disturbing to think this might be a pervasive problem.

      • “Very disturbing to think this might be a pervasive problem.”
        First thing to say is that the topic has wandered. It isn’t an issue with K15, which simply used the same algorithm before and after change, as it has been used for at least five years.

        As to pervasiveness, the place to look is Peter O’Neill’s series of posts. He has done hundreds of them. Some places show quite a lot of flutter, some little, some none. And the flutter does seem to be worse in Australia. It is of course worse as you go back in time, because the adjustments generally hold present constant, so the effect of adjustments accumulates.

        Bates seems to think it is an instability in the code due to bad software practices, or something. I see no evidence of that. The code is open, and the change history shows nothing of it. It is a function of the algorithm.

        The effect doesn’t show up in the global average. I generally deal with unadjusted data, so I haven’t looked much at adjustments to individual stations over time, but I do calculate the global average of adjusted data, and of course it would be there in the NOAA and GISS indices. The GISS code is open, and doesn’t have any steps to deal with flutter. GISS has a history page, with this graph over time:

        https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/01/history_loti.png

        The effect of adjustment is relatively small; the changes in recent years do look significant, even in modern times, but that is mainly due to the effect on the 1951-80 anomaly base.

        The basic trade-off with PHA is that you reduce bias, which could affect the average (but tuns out not to, much) at the expense of introducing unbiased noise, which has much less effect. The fluctuation is a version of noise.

      • Mcintyre has not confirmed the instability.
        The algorithm is supposed to change the estimate of past data as new data comes in.
        This is explained by Thorne.

        That is how it is supposed to work.
        More data in and your adjustments will change in certain cases.

        Instability would require same data in and different outputs.

      • An algorithm that is unstable to what look like small perturbations in the input has to be a worry. More so perhaps than wobbly code that can fixed, since it suggests a rather strange underlying systemic problem with the way the transformations and estimations are being approached.

        In general to have confidence in these kinds of statistic we’d expect progressive convergence as more information was added, particularly since it impacts on the whole time series. It suggests the designers of the algorithms are in some respect weighting the total information in the cumulative history little more than the most recent observation alone. That seems pretty difficult to justify.

      • I have a new post here in which I analyse the variation of GHCN V3 adjusted between now and a file that I have from July 2015. It shows that the differences have a sd over all stations and months since 1879 of 0.33 °C. For the years since 1999, the sd is about 0.22. But the thing is, the mean of differences is very close to zero. I did a breakdown by months, which shows little difference.

        So I think it is indeed part of the trade-off with the PHA, where bias is reduced at the expense of adding noise. But if the noise is unbiased (and this result suggests so), then it is heavily damped by spatial averaging.

      • Mcintyre has not confirmed the instability.
        The algorithm is supposed to change the estimate of past data as

        Change = instability

        4C change in adjustment from 1 year’s additional data?

        Sometimes past adjustment is positive?
        Other times past adjustment is negative?

        Sure looks like something’s goofy here.

  22. The ongoing kerfuffle is itself telling. Karl himself saying the data wasn’t archived in violation of NOAA and Science written policies should have ended it.
    The stonewalled congressional subpoena based on whistleblower input concerning emails; NOAA stomewalled for a reason–they will show the politicization and rush for Paris. Maybe even document Holdren involvement.
    This is the tip of a larger iceberg that is now sinking the ‘good ship CAGW’. Mann’s faulty hockey sticks. Marcott’s paleoclimate misconduct in Science. Deliberately misleading polar bear science exposed by Dr. Susan Crockford. The model/reality tropical troposphere discrepancy that even Santer’s new paper using a (for the tropics) inappropriate stratosphere adjustment could not make go away–still models are 1.7x wrong. Heller’s documentation of GISS data fiddles over time. The shift from Drd946x to nClimDiv to create US state by state warming, documented in essay When Data Isn’t. The artificial warming in global GHCN induced by latitude shift found by Ross McKittrick. The unconstitutional over reach of EPA’s CPP. Renewables induced South Australia blackout. SunEdison and Abengoa bankruptcies.

    • Steven Mosher

      There is no artificial warming in GHCN induced by latitude shift.
      That was debunked long ago.

      You can show this several ways.

      1. In the ISTI database and in the BE database there are 10’s of thousands of stations that are NOT IN GHCN. Judith Misses this point
      Bates Misses this point, Ross misses this point and you miss this point.
      You can, and we have, done versions of the land surface that AVOID
      GHCN ENTIRELY.
      Guess what?
      The answer is the same.
      Choose any subset of GHCN you like. Choose all NON GHCN stations.
      Choose only rural stations.
      Choose only stations not at airports
      Choose multiple randomly selected stations
      Choose A stratified sample ( equal amounts from latitude bins)
      Choose a stratified sample from Every different land class ( crops, grassland, wooded, bareland, etc)

      Guess what?

      The answer is the same.

      Now we are seeing that the argument is that Surface temperature must be handled by the CDR process, a process designed for Non real time
      Satellite data. The UAH CDR is an example I have pointed to many times.
      Yes it is document.
      But
      The documentation is crappy., but it passed Bates Process.
      Do you guys get that? Bates Process doesnt care about the quality of the data one wit. It cares only about checking boxes.
      And worse the UAH CDR its so out of date its too funny

      But since CDRs are now the only currency that counts
      I’ll ask you to provide CDRs or similar for your claims.

      • Geoff Sherrington

        Steven,
        “The answer is the same”.
        At last we agree. Now say after me, ” The answer is the same cacaphony of numerical noise that drowns signals that should or might be there and in the absence of formal and appropriate error analysis to show theveffectsbof noise, permits subjective conclusions suitable for embellishment of social narratives.”
        OK by you? You have been sent relevant examples over the last decade.
        Generally, search “propagation of errors in scientific work Wolfram” for a delightfully clear essay about what should have been standard procedure from the beginning of the current surge of climate style science.

    • “Karl himself saying the data wasn’t archived in violation of NOAA and Science written policies should have ended it.”
      Can’t you ever quote what people actually say? Karl did not say that. And it isn’t true. The data is archived, here. Bates linked to it.

  23. Pingback: Sobre la cacareada “manipulación” de datos de temperatura de la NOAA | PlazaMoyua.com

  24. Judith, it’s all a distraction.
    Apparently meteorologists(and people well educated in climatology) know tomorrow morning is going to be a little above dew point on clear calm nights.

    Do they not see that water vapor is regulating the cooling rate to dew point temperature?

  25. In the UAH example, Bates’ response was “Of course, I never said, use the old one with the known error.” A very reasonable attitude. So when Karl says the old datasets are flawed too, I assume his response would be the same? New datasets will have this correction, if they don’t already. Knowing this, it would be reckless or plain misconduct to use known-to-be-biased data for policy. “Approved” data that is found to be flawed is nevertheless flawed and unfit for purpose. I am not saying anything new here, because skeptics apply the exact same argument to climate models.

  26. We need more technical/scientific whistle-blowers to come out of the woodwork where they have kept their concerns to themselves fearing for their jobs and careers.

    The “Pausebuster” paper rushed through as a political ploy, Mark Boslough attacking Robinson et al. (1998) at scientific conferences for more than a decade with false accusationa of misconduct, Podesta siccing Joe Romm on Pielke Jr. as a political favor — it is finally all beginning to out.

    Time to hit the Restart Button on Climate Science.

    • Steven Mosher

      Looks Like your whistleblower Just recanted

      The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

      • Mosher ==> These types of things need to be allowed the time to sort themselves out — there are and will be whipsawing press reports, seemingly contrary interviews and quotes, journalists of all stripes will emphasize different aspects of the developing story or pull different quotes from the same interviews — status-quo defenders like yourself will grasp at any little thing to vindicate your heroes (or heroic ideas) while extreme skeptic types will prepare effigies for burning in the town square.

        What we need are more whistle blowers to come forward out of their comfort zones, out of under the fear of losing their jobs or risking their careers, and speak out about whatever they know concerning this and other possible biases and questionable scientific behavior at NOAA (and other global warming activist government agencies).

        When enough has come out, when enough investigation has been done, we can have a clearer idea of the severity of the problem — if indeed there is a problem.

        It is childish to think that some one new interview with Bates will make it all better.

      • > It is childish to think that some one new interview with Bates will make it all better.

        It’s dishonest to reframe Steven’s point in that manner, Kip.

        Go Team Integrity ™.

      • Bates ==> Mosh says: “Looks Like your whistleblower Just recanted” …”Bates said in an interview Monday with The Associated Press”.

        My point is that quoting different parts of different fresh interviews by different journalists at different news outlets does not make the overall issue go away….all those taking sides can find and pull quotes to suit themselves.

        Bates, of course, has not “recanted” …. he hasn’t even presented his evidence, which he says he has carefully collected over time, to anyone yet — so we can’t really know what is there.

        So, allowing time for the facts to come out in their due time is necessary.

        Bandwagoning to defend Mosher’s defence of [hard to tell what he is defending, he seems to be “just defending”….] hardly adds to the conversation.

      • Actually Brandon we can simply ignore Mosher’s repeated posting of the same comment about Bates recanting. On the grounds that Mosher has apparently lost the ability to understand the meaning of words.

        Bates’ original phrase “a thumb on the scale” is rather broad and can cover a range of actions. The only thing his second interview shows is a clarification. In that Bates did not mean to accuse Karl of tampering with the data or misconduct. Actions which could be included under his original phrasing. That is not recanting. If you and Mosher don’t believe that people in supervisory or management positions can influence what happens under them, then you must live in a bubble. As I mentioned above, influence can take many forms and does not have to be illegal or unethical. I believe that Karl did not do anything illegal or unethical. However it is important to know if he failed to follow established processes and if there were political or policy issues impacting the research.

  27. Zeke Hausfather tweets a question in rebuttal to one of Bates’ claims:
    https://twitter.com/hausfath/status/828634322357805056

    “Also, how is the data NOAA provided on the FTP site “non-machine readable”? My machine read it, and Berkeley uses same formats…”

    Does anyone have an answer?

    • I fed it into my toaster… nothing.

    • Yes, the “machine readability” is a pointless nit-picking complaint. Presumably he thinks it should be in NCDF or some such. The GHCN and ERSST data are on the archive in exactly the ascii format they are regularly made available, and which my computer reads every day.

      If a machine can’t read this data, who can? No-one types it all out. A machine put it there.

      • > If a machine can’t read this data, who can? No-one types it all out. A machine put it there.

        If a machine can do Gödel’s numbering, then it’d be possible to have an automatized undecodable system.

        Ask DavidW, Denizens’ logic guru, about that one.

      • Nick,

        “If a machine can’t read this data, who can? …A machine put it there.”

        I don’t have any knowledge or experience with NOAA’s data archives. But government agencies often deliberately post material in non-machine readable formats. Most commonly, jpegs of documents. Sometimes low-res jpegs, so optical character resolution tools can’t reliably read them.

      • Nick, I believe the “machines” would be the ones used by the US government in an official capacity. You know, outdated, non-user friendly relics of a by gone era using limited software.

      • Editor,
        “Most commonly, jpegs of documents. “
        That is certainly not the case here. The complaint seems to be about the use of normal ascii. As Zeke says, he had no trouble using it. Nor do I.

      • Bates must have meant EBCDIC (Extended Binary Coded Decimal Interchange Code), a relic from the 1960s when COBOL was the language djour of bureaucrats like him.

      • jack, “Bates must have meant EBCDIC (Extended Binary Coded Decimal Interchange Code), a relic from the 1960s when COBOL was the language djour of bureaucrats like him.”

        I think there have been some advancements :)

      • Nick,

        “That is certainly not the case here.”

        As I said, it’s not something I have any knowledge of. I just pointed out that the general assertion you gave was incorrect. Data can be posted in a fashion almost impossible for the average user (i.e., not NSA) to convert to machine-readable form.

        It wouldn’t dent your climate warrior ethic to admit the occasional mistake or over-statement.

      • Editor,
        “I just pointed out that the general assertion you gave was incorrect. “
        It wasn’t an assertion that it is impossible to post something that is not machine readable. I was specifically referring to the ascii data on the archive.

    • Steven Mosher

      I think Bates means “Self Documenting” like NetCDF or HDF which is used by him and others in the satellite domain.

      Frankly NetCDF and HDF make my job a hell of a lot harder . HDF is a nightmare to use. I cannot imagine using either of these for temperature series, Simple Human readable csv files make the data easy for anyone even citizen scientists easy to use.

      Bates is actually suggesting formats that make it MORE DIFFICULT for citizen scientists to ingest data

      In the past Human readable used to refer to plain text ascii data.
      You could look at it and read it.

      Machine readable meant the data was encoded so that only a machine
      could read and understand.

      Machine readable sucks. Yes when storage and bandwidth was expensive you would often revert to Machine Readable ( for example compress the data ) so reduce your footprint.

      NetCDF and HDF would be ok for Gridded data, but for Time series data,
      It is hitting a fly with a sledge hammer

  28. To be succinct, here is my response to one of Zeke’s comment’s with respect to a subject I have expertise:

    “Zeke has written a Factcheck on David Rose’s article. His arguments are that….

    ‘NOAA’s land surface temperatures are similar to other data sets'”

    The analyses from the separate groups are NOT independent. They draw from mostly the same raw data and/or, in the case of BEST, from the same geographic areas. The systematic warm bias and other uncertainties that we have identified in our peer reviewed papers permeate all of their analyses.

    Bottom line: NOAA’s land surface temperatures are similar to other data sets because they work from mostly the same data!

    Roger Sr

    • Roger,

      It is both true that not all the groups are independent (i.e. largely rely on the same set of land and ocean measurements), and true that the fact that other groups get substantially the same results as NOAA provide a good independent validation of NOAA’s adjustments to and processing of the data.

      • Zeke- It might be an independent choice for the adjustment process and processing of the data, but this does not make them independent assessments of the robustness of the anomalies and trends. It is the later as to how the separate analyses are presented to policymakers and the media.

        That is not an accurate and honest characterization. They are closely interrelated assessments.

        Roger Sr

      • I think we largely agree on that point. But folks seem convinced that somehow NOAA is dramatically changing what we (and other groups) see when they look at the raw data over the past few decades, which is decidedly not the case.

        http://www.popsci.com/sites/popsci.com/files/styles/large_1x_/public/images/2017/02/adjusted_vs_raw_noaa.jpg

      • If anyone looking at either of those data series concludes that they are anything but representations of minor warming, with fits and starts, I’ll be happy to entertain an alternative view. Both data series must be truncated at the end of 2013, however, to avoid any erroneous conclusions based on weather related to the Blob and super El Nino.

      • Zeke,

        I’d rather say that the datasets are ontologically dependent but that the researchers achieved some kind of epistemological independence.

        Auditors usually like to pay due diligence to second one, while quarrelers usually prefer the raise concerns about on first one.

      • Zeke, can you compare ERSSTv4 to the original ERSST? Will you be able to compare ERSSTv4 to ERSSTv32? What policy recommendations based ERSSTv2 need to be modified due to new information available with ERSSTv4? Based on currently available information, what is the most probably climate sensitivity and how does that compare to previous estimates? Is there a trend in climate sensitivity best estimates since 1980 that might be usefully considered in adjusting climate policy?

      • Geoff Sherrington

        Zeke,
        The similarity of final results between groups might be an accident from all groups using similar reasoning and making the same mistakes at the same places. For example, if all use pair-wise comparisons of close stations, an anomaly at one station can also be seen at the other if they are both due to a shared weather event; or, less easy to deal with, if they have a regional electrical power supply glitch that puts an artefact into each, one that has a chance of being detected as a weather event. In general, there are weather and non-weather events that can lead to pair wise adjustment and the software used to distinguish between them is not error proof.
        Many problems can be hypothesised. The exact shape of the temperature/voltage curve for electronic sensors varies with the maker and can be substantially different. Pairwise work can meet a difference, then be required to make a choice to adjust up or down. This choice is not real because the correct way is unknown so roughly 50% of the adjustments will be wrong.
        Two of the bigger problems arise from subjective conventions. Maintaining the present and adjusting the past; and generally having the weight of more data emerge with cooled early years. If all parties treat these similarly, and start with similar data sets, then it would be cause for comment if they did not agree. But agreement is NOT the same as correctness.

      • Zeke – There are also sites with large deviations between raw and adjusted.

        What I would like to see, and recommended before I was forced off of the CCSP 1.1 Committee, was that the change at each step in the adjustment from the raw data for every site be quantitatively reported. This is before any grid averaging,

        Also, Tmax and Tmin separately. Even with the raw data, there is the systematic warm bias in miniumum temperature over land when it is to be used as part of the diagnosis of global warming, There will even be a warm bias in maximum temperature in locations such as the winter Arctic where the surface layer is often stably stratified.

        Roger Sr

      • Mornin’ Roger (UTC),

        Since you bring up the matter, do you have any strong views on the question I asked John J. Bates about Arctic temperatures in these hallowed halls not so very long ago?

      • Why does Zeke keep showing the whole global record when the authors of Karl et al admit the greatest impact of their changes was in the latest decades of the modern period? To a technical audience that knows the debate around the instrumental record, this makes him look foolish.

    • Dr Pelke, I am not a scientist. With so many data sets and papers being written from the same instrumental and paleo data with conclusions and assertions that align with the authors CAGW and political views how does anyone make sense of what the climate is or isn’t doing? It appears that most papers are simply statistical re-analysis of the same data hoping to find evidence to support their arguments. There seems to be so much that we don’t know but look at all the time and money that is spent trying to dis-prove the pause with statistical approaches of the same data. This argument is no different. I have followed your posts and I find you a reasonable player. So, Dr. Peilke, Is this science? I hope so but it looks more like a school yard fight among egos.

      I saw multiple groups do approximately the same thing, and they all got about the same results. I decided to use my work skill to look at what the stations actually measured, to do it myself. At least I know I where the warts are. The results are it’s not co2, it’s water.

  29. So this is not a ‘he said’—‘the other he said’ situation.

    Clearly not. He’s tried to dance around it, as have you, but Bates has confirmed what Thorne said: Bates wasn’t involved in any technical discussions of ERSSTv4 or ISTI. Having ‘insight into how code ran’ on previous projects is really besides the point.

    Response: Thorne has not been on site (NOAA NCDC) for years and not during the final few years when this took place.

    This is a statement of fact, meaning you have clear evidence that the final ERSST v4 record shows a significantly warmer recent trend than the working record at the time Thorne left…Yes?

  30. Chronology of the game from the cheap seats:

    Skeptics – “there is a pause”
    Non-skeptics – “the pause is a myth”
    Karl 2015 – “ok, there was a pause but it goes away after we fix the data, trust us”

    I hear dragging sounds interspersed with loud kicking and screaming.

  31. The ‘whistle blower’ is
    John Bates…

    –i.e., sort of like the source for 2009foi.pdf CRUgate…?

    whis·tle-blow·er
    /ˈ(h)wisəl ˌblō(ə)r/

    noun: whistleblower

    a person who informs on a person or organization engaged in an illicit activity. [emphasis added]

  32. Peterson: “Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.”

    Thorne “Whether the ships are matched to buoys or buoys matched to ships will not affect the trend.”

    Both these statements are correct, apart from possible ‘edge effects’. It is basic calculus. But that is not a free pass to all the other fanciful arguments defending Karl’s adjustment.

  33. Peterson: “Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.”

    This statement and similar from Thorne about the trend are correct, apart from possible ‘edge effects’. It is just basic calculus. But that is not a pass to more fanciful extensions of this idea being used to defend Karl’s adjustment.

    • But that is not a pass to more fanciful extensions of this idea being used to defend Karl’s adjustment.

      I don’t know what this is meant to mean? And they’re not Karl’s adjustments anyway. They were published in Huang et al. 2015, on which Karl was not involved.

      • Nick ==> Some other time we’ll have something about “trend-focused, anomaly-focused” data — not today, I’m busy with something else.

    • Peterson: “Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.”

      What you don’t end up with is a more accurate numerically correct temperature data set — forcing the more-accurate buoy data set to agree with the less-accurate ships data set is simply “wrong” — it adulterates the overall with data set with less-accurate information. It is only “right” if one does not care about accuracy but is seeking only a politically-useful trend-focused answer.

      • “but is seeking only a politically-useful trend-focused answer.”
        Absolute nonsense. As a matter of simple arithmetic, it makes no difference to any published number. Absolutely none. How can that be “politically useful”.

        The adjusted numbers are only used in preparing anomaly averages.

      • Nick ==> Some other time we’ll have something about “trend-focused, anomaly-focused” data — not today, I’m busy with something else.

  34. The responses to Peter Thorne’s comments assume Ireland is on another planet and Prof. Thorne in purdah. Bates is simply being foolish, but Eli repeats himself.

  35. The gruel is thin indeed

  36. …failed to follow… guidelines established by Science for publications.

    Science disagrees:

    https://twitter.com/climpeter?ref_src=twsrc%5Etfw

  37. Peterson: “Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.”

    Emmm?

    Above it says the buoys are weight x7 compared to ships.

    Ships and buoys cover different areas of the ocean.

    Are we expecting the data is interchangeable without any resulting difference?

    Seems a tall story.

  38. Pingback: US govt agency manipulated data to exaggerate climate change – whistleblower | |

  39. Pingback: US govt agency manipulated data to exaggerate climate change – whistleblower - You-Blog.Club

  40. the process of establishing credibility for climate data sets

    Ok, let’s talk about process…

    Reading the previous post it seems that the ERSSTv4 went through all the correct checks and processes, and was officially approved as an operational product. Right? The only concern expressed was that release of the data was delayed until after publication of Karl et al. 2015 despite it being available months earlier (which, by the way, flies in the face of the ‘rushing’ narrative). Regardless of the rights and wrongs of that, it has no implications from a scientific integrity or dataset building process perspective.

    To sum up, it appears to me unequivocal that the building of the ERSSTv4 dataset has demonstrated clear scientific integrity in accordance with the standards written about in these posts… and yet it’s the SST record which seems to be the primary focus of ire. That’s difficult to square with insistence that process is what this is about.

    You’ve even gone so far as to publish results from a non-peer-reviewed, non-internally-checked, non-operationally-ready SST record to support claims of problems with ERSSTv4. How is that for process integrity?

  41. There would seem more fundamental issues than ships versus floating buoys. Although one might wonder which is accurate.

    1, There are no long term trends. Ther are multidecadal trends that are driven by shifts in the Pacific state – and a rate of greenhouse gas warming of O.032W/m2. This is set against changes of +2/-1 W/ms in net toa radiant flux.

    https://watertechbyrie.files.wordpress.com/2014/06/ceres_ebaf-toa_ed4-0_anom_toa_net_flux-all-sky_march-2000tojune-2015.png

    The multidecadal climate shifts can be distinguished in changes in ENSO activity – or with network math.

    http://onlinelibrary.wiley.com/doi/10.1029/2008GL037022/full

    The last shift happened after 1998. The next is due within a decade. Pre 2002 data is far less reliable than post – however, ISSCP-FD and Project Earthshine did capture an increase in Earth albedo at the turn of the century. On energy considerations, I am expecting a cooler trend since the turn of the century – but we can’t know – short record and considerable noise – until climate shifts again. Where it shifts to is the interesting question.

    2. The surface records miss latent heat. This is such a simple physical fact – temperature is defined by the WMO as sensible heat. It misses a huge and variable part of the global energy budget. Latent heat flux changes considerably with changes in soil moisture.

    http://www.nature.com/nature/journal/v491/n7424/images/nature11575-f1.2.jpg

    3. Aqua and Argo have provided unprecedentedly precise data since earlier this century. It is time to go with 21st-century technology.

    I don’t understand the point of any of this. There are far broader scientific considerations. I don’t know where any of this is going – I have an idea of where it should be going. Warning – strong language.

    https://watertechbyrie.com/2017/02/06/the-climate-problem-and-the-solution/

    • Excellent point –

      “The surface records miss latent heat. This is such a simple physical fact – temperature is defined by the WMO as sensible heat.”

      See

      Pielke Sr., R.A., C. Davey, and J. Morgan, 2004: Assessing “global warming” with surface heat content. Eos, 85, No. 21, 210-211.
      http://pielkeclimatesci.wordpress.com/files/2009/10/r-290.pdf

    • ” temperature is defined by the WMO as sensible heat”
      No, they define temperature as everyone else does. And sensible heat is defined as heat (transfer) you can measure with a sensor (thermomter, thermistor, finger). We have over a century of thermometer readings. What do you suggest?

      • I checked – I scanned the WMO manual – which as a hydrologist I have read long ago. Maybe I was thinking there was some hint of estimating energy in water vapour. But of course even measurement of soil moisture is a fairly modern innovation and not widely adopted.

        I am saying quite clearly that past data is what we have – but the future is satellites.

        And yours is a colossally inane quibble.

      • And yours is a colossally inane quibble.

        It’s a colossally inane quibble?

        The WMO uses the same definition of temperature as everyone else. The same definition as physicists, chemists, high school kids in science classes. Heat and temperature are two different things; two different concepts.

      • Thermometers measure sensible heat. You should access a primer if such elemental physics are beyond you.

  42. Pingback: US govt agency manipulated data to exaggerate climate change – whistleblower | TrumpShake.com

  43. richardswarthout

    Is it necessary to grapple in the grass regarding the work of NASA. Look only at their public statement regarding the 0.04C increase between 2015 and 2016:

    It’s stuff like GISS head Gavin Schmidt telling us that “this year was ridiculously off the chart,” or NOAA’s Deke Arndt telling us “We’re punching at the ceiling every year now, that is the real indicator that we’re undergoing big changes.”

    Richard

    • This is NASA’s work on temps.

      http://www.drroyspencer.com/wp-content/uploads/UAH_LT_1979_thru_January_2017_v6-550×317.jpg

      http://www.drroyspencer.com/wp-content/uploads/UAH_LT_1979_thru_January_2017_v6-550×317.jpg

      It captures the energy content of the troposphere. Since Aqua in 20022 it is precise and reliable.

      Surely point to the difficulties in the surface record – but also to the much better data available. Why settle for scrabbling in the dirt when you have a God’s eye view?

      • So good in fact that I had to repeat it.

      • Steven Mosher

        Actually that is not NASA that is UAH

        Having read the CDR I can assure you that

        A) it is heavily adjusted and tweaked
        B) The code referenced by the CDR has multiple bugs
        C) There is no attempt to calculate structural uncertainties.

      • The satellite was launched by NASA – who employed two groups to crosscheck temperature results.

        Another colossally inane quibble Mosh?

      • The AQUA satellite was launched by NASA.

        ‘Roy W. Spencer received his Ph.D. in meteorology at the University of Wisconsin-Madison in 1981. Before becoming a Principal Research Scientist at the University of Alabama in Huntsville in 2001, he was a Senior Scientist for Climate Studies at NASA’s Marshall Space Flight Center, where he and Dr. John Christy received NASA’s Exceptional Scientific Achievement Medal for their global temperature monitoring work with satellites. Dr. Spencer’s work with NASA continues as the U.S. Science Team leader for the Advanced Microwave Scanning Radiometer flying on NASA’s Aqua satellite. ‘

        Two groups were contracted to provide independent results as a crosscheck.

      • Mosher: so if you are right, NASA launches satellites, takes data, and doesn’t make sure it’s processed properly? And this has gone on for years? Who was in charge? I want to suggest a demotion.

      • “A) it is heavily adjusted and tweaked
        B) The code referenced by the CDR has multiple bugs
        C) There is no attempt to calculate structural uncertainties.”

        Yes Steven and also …
        It is also a massive outlier and does not correlate with RATPAC “A” sonde data, since the MSU to AMSU change-over in ’97.

        http://postmyimage.com/img2/995_Tropospheretrends.png

        http://postmyimage.com/img2/792_UAHRatpacvalidation2.png

        This is how RSS has dealt with the problem … but it still exists…..
        https://tamino.files.wordpress.com/2016/04/diff.jpeg

        In short, by any reasonable measure, it cannot be taken as a reliable metric.

      • Yes and which urban doofus hipster echo chamber did you steal that from?

        Radiosondes? Yeah right.

      • “Yes and which urban doofus hipster echo chamber did you steal that from?
        Radiosondes? Yeah right.”

        Yes, you know, err – those things that go up from hundreds of stations twice a day, all around the globe and that give actual thermometer readings of temp/hum (and other elements) as they ascend.

        https://en.wikipedia.org/wiki/Radiosonde

        Each will have some error, but the RATPAC “A” series….

        https://www.ncdc.noaa.gov/data-access/weather-balloon/radiosonde-atmospheric-temperature-products-accessing-climate/ratpac-a

        would average them out.
        You can do that with multiple sensors you see, as apposed to the one MSU/AMSU sensor.

        “From 1958 through 1995, the bases of RATPAC-A are on spatial averages of the Lanzante et al. (2003; hereafter LKS) adjusted 87-station temperature data. After 1995, they are based on the Integrated Global Radiosonde Archive (IGRA) station data, combined using a first difference method (Free et al. 2004). For analyses of interannual and longer-term changes in global, hemispheric, and tropical means, the NOAA team recommends using RATPAC-A because it contains the most robust large-scale averages. Access to the RATPAC-A data files is available on the FTP site.”

        Oh, and don’t mention it.
        Glad to have lifted your ignorance just a tad my friend.
        BTW: could please elucidate a better way to calibrate tropospheric temp data?
        Aside from having Santa Claus pass us down some data every night of the 24/25th Dec

      • Each will have some error, but the RATPAC “A” series….
        https://www.ncdc.noaa.gov/data-access/weather-balloon/radiosonde-atmospheric-temperature-products-accessing-climate/ratpac-a
        would average them out.

        The method involves taking the difference in temperature between one time step and the next (the “first difference”), then computing large-scale means of the FD series, and finally reconstructing large-scale temperature series from the FD series.

        This is basically how I process my surface station data, where the steps are each days measurements, and I can exclude stations that are not continuous series for a year to reduce the random error as identified below.

        Omitting portions of the station time series around the times of known changes in instruments or procedures, is an attempt to eliminate the effect of inhomogeneities due to such changes. However, the method introduces a random error that increases with the number of time gaps in the data and increases with decreasing number of stations, so the results are limited to large-scale means.

      • ‘Now for the important part: How are these instrument digitized voltages calibrated in terms of temperature?

        Once every Earth scan, the radiometer antenna looks at a “warm calibration target” inside the instrument whose temperature is continuously monitored with several platinum resistance thermometers (PRTs). PRTs work somewhat like a thermistor, but are more accurate and more stable. Each PRT has its own calibration curve based upon laboratory tests.

        The temperature of the warm calibration target is allowed to float with the rest of the instrument, and it typically changes by several degrees during a single orbit, as the satellite travels in and out of sunlight. While this warm calibration point provides a radiometer digitized voltage measurement and the temperature that goes along with it, how do we use that information to determine what temperatures corresponds to the radiometer measurements when looking at the Earth?

        A second calibration point is needed, at the cold end of the temperature scale. For that, the radiometer antenna is pointed at the cosmic background, which is assumed to radiate at 2.7 Kelvin degrees. These two calibration points are then used to interpolate to the Earth-viewing measurements, which then provides the calibrated “brightness temperatures”…

        One can imagine all kinds of lesser issues that might affect the long-term stability of the satellite record. For instance, since there have been ten successive satellites, most of which had to be calibrated to the one before it with some non-zero error, there is the possibility of a small ‘random walk’ component to the 30+ year data record. Fortunately, John Christy has spent a lot of time comparing our datasets to radiosonde (weather balloon) datasets, and finds very good long-term agreement. http://www.drroyspencer.com/2010/01/how-the-uah-global-temperatures-are-produced/

        Satellites are not calibrated to radiosonde data – although apparently comparisons were made.

        http://www.remss.com/measurements/upper-air-temperature/validation

        I have suggested before that you get out of the blogs because they are always wrong. And still you miss the main point.

      • Steven Mosher

        You guys Missed Judith’s memo

        You need CDR or you have to shut up.

        This applies to everyone ( She exempts herself of course )

      • I don’t need CDR to quote verbatim on calibration from Christy and Spencer. Christy and Spencer do. It is at any rate a very uninteresting discussion. As are bloggists who ‘want to take the piss out of Spencer’ for a presumed lack of coding elegance and efficiency.

        What we generally lack is a balanced appreciation of the primary sources – rather than a cdr.

      • Steven Mosher

        Launching a satellite does not produce a data product.

        Any way you cut it UAH version 6 is not a NASA data product.
        It has not been through Bates process.
        Judith will not approve it as data that can inform or shape policy and regulation.

        She just shot Spencer in the head

      • By Wandering in the Weeds you miss the point again, Mr. Mosher.

        Dr. Curry was simply pointing out there were other creditable scientific views that countered the CAGW meme. She was not asserting that her satellite-derived data series met some obscure standard of validation methodologies. She was indicating that extreme policies were unwarranted.

        IPCC climate models have proven to be bunk over a 40-year time period. Just one example: No tropical troposphere hot spot.

  44. Back in 2009, Steve Mosher had this to say over on The Blackboard concerning corruption in science:

    Enemies caught in action!
    The Blackboard
    21 November, 2009 (10:35)

    steven mosher (Comment #24083)
    November 21st, 2009 at 12:29 pm

    I think the bottomline on this whole story is this. The institutions that govern scientific behavior have gone awry. Those institutions have been corrupted by money and power and politics. The tonic for this is transparency. Free the data; free the code; free the debate.

    But the AGW side is interested in controlling the message. They fight against data release because they fear what people will do with it. They fear that data will be misused: it will be. They fear people finding errors: errors will be found. They fear that people will be less certain: they will be. And they fear that it may take a long time to convince people to take action: it will. And so they act out of fear and try to control the message. Everyone who understands the nixon whitehouse understands how this fear drives people to do crazy things. The one thing they never feared: disclosure. Leaks. And so the thing they feared the most, delaying action on climate science, is the very thing they may have got. They should have trusted that open debate would yield the next right action in the shortest time possible. They didnt. They feared a “corporate enemy” that would delay action. And ironically in the end they ended up being the thing they feared.

    Are Steve Mosher’s words from 2009 as appropriate today as they were seven years ago?

    • Steven Mosher

      What you fail to realize is that….since 2009
      we have pretty much got everything we asked for.

      That’s right

      ALL the NOAA data
      The NOAA adjustment code..

      ALL AVAILABLE NOW.

      The Issue here is that Bates is complaining because they didnt follow his CDR process which was developed for satellite data.

      A Process, By the way, which yeilds “approved data” That is YEARS out of date.

      See Lief Svalgaards Critcism of the CDR that Bates praised as the Best.

      We Got the NOAA data for K15 right from the start

      Bates problem?

      It wasnt posted in The FORMAT HE LIKES !

      If you followed everything I wrote, I was very clear. When we demand data and code, we DONT DEMAND that you serve it to us in a FORMAT WE DEMAND. we will do the work to ingest any data in any format you choose. we just want the data.
      We dont demand that your code be commented or pretty or tested, or bug free. We just want the code AS IS, as it was run. We will figure out how to run it

      The whole POINT was to Free the scientist from the ENGINEER JOB, Not to demand as Bates does, that scientists also become engineers.

      Give us the data, give us the code. we will figure it out without pestering you. and most importantly we promise not to bitch about your engineering or coding practice.

      • So the institutions gone awry and corrupted by money and power and politics are all ok now?

        Gimme a break!

      • Steve Mosher: …… Bates problem? …… It wasn’t posted in The FORMAT HE LIKES ! …… The whole POINT was to Free the scientist from the ENGINEER JOB, Not to demand as Bates does, that scientists also become engineers. …… Give us the data, give us the code. we will figure it out without pestering you. and most importantly we promise not to bitch about your engineering or coding practice.

        A portion of my thirty-five year career in nuclear construction and operations was spent as a consultant performing nuclear-grade Quality Assurance (QA) and Quality Control (QC) activities in support of plant operating license applications. During that period, I was contracted for a time to supply QA/QC consulting services for the government’s high level nuclear waste repository program.

        The way it works in DOE’s nuclear waste repository program is that software and data are QA/QC-ed according to a graded approach. The highest level of attention is given to software and data directly employed by those analytical products which support an NRC license application.

        A specific run of a software system which processes one or more specific sets of data — all of which together support some specific analytical product which in turn supports a technical decision or a scientific conclusion — are all considered to be One Thing.

        In other words, for work that directly supports an NRC license application, the software code, the data, and the analytical results are not separate items which can be independently assessed. The QA/QC process extends to looking closely at how they were actually employed together in producing a specific analysis product which is being cited in the license application.

        The QC stamp of approval does not go on the analytical product as a whole unless: (1) All of its separate components have been produced according to sound software engineering and data management practices; (2) The provenance of the data, and any caveats that might apply to it, are documented in the project record; (3) The QA/QC process that was applied to the software and data is documented in auditable form for the specific software runs that were made; and (4) The specific runs of the software, and the specific data the software processed, are all archived in machine readable form and can be easily recovered and rerun if the need arises.

        Doing it this way is an expensive and time consuming process for the people who produce the work. More often than not, QC-trained software engineers must assist the scientists in developing, testing, QA/QC-ing, and archiving the software and the data according to the project’s QA/QC standards.

        Why are things done this way in the DOE high level repository program?

        It’s because the consequences of doing an unprofessional job are unacceptable if errors and mistakes result in a rejection of the license application by the NRC; or what is much worse, the errors and mistakes result in the failure of the repository itself to perform as expected.

        Is the work that NOAA is doing in the area of ocean and atmospheric temperature trend assessment of such importance that a robust software and data QA/QC program ought to be applied to it? And if so, to what extent; i.e., should NOAA apply the same rigorous QA/QC standards to its work as DOE does to the nuclear waste repository program?

      • Mr. Mosher, by Wandering in the Weeds you miss the main point: IPCC climate models are inadequate for the purposes of fundamentally changing our societies, economies and energy systems.

        Debating tenths (even one hundredths!) of a degree still does not resolve the inconvenient truth that our planet is not warming where and to the extent predicted by the best “science” available. The data I have seen simply show a slow and uneven warming following the Little Ice Age. No accepted theory accounts for numerous decadal bursts of estimated warming and cooling.

        CliSci is reduced to propping up a failing meme with desperate misdirection. Additionally, no data set can be used to predict the future, as many are wont.

      • Steven Mosher

        “Mr. Mosher, by Wandering in the Weeds you miss the main point: IPCC climate models are inadequate for the purposes of fundamentally changing our societies, economies and energy systems.”

        Argument by assertion.
        Here is my counter argument.
        ” IPCC climate models are adequate for the purposes of fundamentally changing our societies, economies and energy systems.”

        How can I make this argument? Simple. IF you look at the model data
        and observations and PROPERLY mask the model outputs so that
        they match the spatial distribution of the observations, the difference
        between models and observations shrinks. The models are closer than you think. Second, A model that is biased High CAN STILL INFORM
        Policy. The models predict .2C of warming per decade. They appear to be biased high against observations that show something on the order
        of .15C to .17C. Any policy maker can take this into account.
        Policy does not need an Optimal answer or exact answer to proceed.

      • The models are closer than you think.

        The models are wrong, the attribution is wrong.
        It’s the ocean cycles, and water vapor distribution.

      • Actually, Mr. Mosher, not adequate. Just one example:

        AR5 reduced climate model “projections” for the next 20 years based on some unknowable “expert opinion.”

      • Oops! Make that 30 years based on 2005 model hindcast cutoff.

      • Steve Mosher writes- ” IPCC climate models are adequate for the purposes of fundamentally changing our societies, economies and energy systems.”

        My response- Your conclusion is deeply flawed.

        Can ANY model used by the IPCC be relied upon to determine how the weather/climate will change in any location over the next several decades?

        Answer- NO.

        GCMs provide no reliable information of how the key components of weather at particular location will change. This means that policy makers use historical weather data and population change projections regarding what changes in local infrastructure will be needed over time.

  45. It’s never too soon to start the smears when defending against a challenge to climate change politics!

    See “Article names “whistleblower” who told Congress that NOAA manipulated data” by Scott K. Johnson at Ars Technica — “Allegations in a Daily Mail article seem more office politics than science.”

    There is nothing in the article even attempting to support the claim in the title.  It repeats the rebuttals described in this post, all accepted credulously (despite some being from participants in the debate). Also note this office gossip (much more so than the serious process violations alleged by Bates):

    “There may also be something beyond simple “engineers vs. scientists” tension behind Bates’ decision to go public with his allegations. Two former NOAA staffers confirmed to Ars that Tom Karl essentially demoted John Bates in 2012, when Karl was Director of NOAA’s National Centers for Environmental Information. Bates had held the title of Supervisory Meteorologist and Chief of the Remote Sensing Applications Division, but Karl removed him from that position partly due to a failure to maintain professionalism with colleagues, assigning him to a position in which he would no longer supervise other staff. It was apparently no secret that the demotion did not sit well with Bates.”

    • ” this office gossip (much more so than the serious process violations alleged by Bates)”
      A lot of what Bates said is “office gossip”. Some rather smearish. For example:

      “One of the co-authors said there were ‘some decisions [he was] not happy with’. “

      “A NOAA NCEI supervisor remarked how it was eye-opening to watch Karl work the co-authors, mostly subtly but sometimes not, pushing choices to emphasize warming.”

      “Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets—in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.”

      None of this is substantiated. The last speaks of evidence, but none is provided.

      • Nick,

        I agree (but I didn’t see it until you pointed it out). I’ll state that more strongly: editing out the office gossip would have made Bates’ article stronger — with a clearer focus on his serious claims of process violations.

        Still, it is odd that the title of the Ars Technica article makes a claim that’s not even mentioned in the article. It’s like that classic “Bloom County” strip by Berke Breathed:

        Bedfellow: “Hello, Bloom Beacon! This is Senator Bedfellow! What’s with this headline? There’s no story, just a headline!”

        Milo: “Which headline?”

        Bedfellow: “The big headline on the front page!” ‘BEDFELLOW: THE SECRET LIFE OF A WIFE-SWAPPING ATHEIST’”

        Mile: “Oh, that’s just a typo.”

      • Fabius,

        I believe this is an example of assumed prior knowledge – that Lamar Smith has previously claimed to have an NOAA-insider “whistleblower” supporting his crusade against Karl et al. 2015. The reasonable assumption is that Bates is the guy Smith was talking about. It’s tying this story in with previous stories.

      • Paul,

        “I believe this is an example of assumed prior knowledge”

        I believe the technical term for the Ars Technica headline is “making stuff up.”

        If this was an actionable statement and they were in court for libel, I doubt giving “making stuff up” a fancy name would be a viable defense.

      • I believe the technical term for the Ars Technica headline is “making stuff up.”

        Seriously? Lamar Smith claims he’s being fed info by an NOAA-insider, then someone just retired from the NOAA comes out and starts making essentially the same allegations Smith has been making… and you think it’s a stretch to assume that’s the NOAA-insider? Unless you’re assuming Smith was making it up?

      • Paul,

        “and you think it’s a stretch to assume that’s the NOAA-insider? ”

        I believe that kind of guessing in headlines is not how actual journalists write. It’s a tabloid headline that like those in the Daily Mail.

        Worse, it was stated as a fact. Even your confident guessing can’t make it a fact.

        Bates distributed a memo about this to the NCEI Science Council, and says that there was discussion among members about it. It was discussed openly at one meeting. And that was just the last round of debates about Karl 2015. This was not secret information.

        There are 11,000 people at NOAA. Quite a few were involved or heard about the discussions about Karl’s paper, and probably some non-NOAA employees.

  46. Pingback: US govt agency manipulated data to exaggerate climate change – whistleblower | tomfernandez28's Blog

  47. Zeke deke’s it:

    [ehak] I think Dr Curry is however saying it would change the trend

    [Nick] But she is wrong, and Peterson is right. It is a matter of elementary arithmetic.

    [Judy] Nick, the new ERSSTv5 paper finds a 0.07C difference.

    [Zeke] Judy, the change in ship-buoy differences in ERSSTv5 is not due to the choice to adjust the ships down to the buoys (vs. buoys up to the ships). Rather, its due to the use of a dynamic buoy-ship difference calculated for each month by comparing coincident ship and buoy measurements. This change reduces the average ship-buoy difference from the 0.12 C assumed by ERSSTv4 to 0.08 C. How this is applied (ships down by 0.08 C or buoys up by 0.08 C) would have zero effect on the resulting anomalies.

    Overheard: https://judithcurry.com/2017/02/06/response-to-critiques-climate-scientists-versus-climate-data/#comment-837228

  48. My question is why did it take so long for someone to develop a conscious? These are the challenges these climate “scientists” will likely face in court. It won’t be pretty. The “science” it a pure fraud.

    Climate “Science” on Trial; The Smoking Gun Files
    https://co2islife.wordpress.com/2017/01/17/climate-science-on-trial-the-smoking-gun-files/

    The one good thing that is likely to come from all this is that the Climate Bullies will be knocked off their high horse.

    Climate Bullies Gone Wild; Caught on Tape and Print
    https://co2islife.wordpress.com/2017/01/22/climate-bullies-gone-wild-caught-on-tape-and-print/
    https://co2islife.files.wordpress.com/2017/01/c53234bb1ea84a16f8e9e351ef1d136c.jpg

  49. BTW, the answer to all this corruption is simply an open source temperature reconstruction. No open source organization would accept the garbage that goes on behind closed doors at NOAA and NASA. They are staffed by like-minded activists, as this and other article prove. “Adjustments” like Mike’s Nature Trick to Hide the Decline are simply unique to the researcher that wants to create an outcome. That is fraud, not science. More transparency is needed, and in the computer and sharing age, that shouldn’t be difficult.

    Climate “Science” on Trial; The Consensus is more Con and NonSense than Science
    https://co2islife.wordpress.com/2017/01/29/climate-science-on-trial-the-consensus-is-more-con-and-nonsense-than-science/

    • Steven Mosher

      Sorry, we did that.

      In the beginning we published our entire code base in SVN so that folks could install it and run it for themselves.

      Result?

      1. One guy argued that we didnt post the password ( it was in the file he couldnt bother to read)
      2. Other people complained we used MATLAB and wanted me to port it to OCTAVE.
      3. Some people complained that we needed to release all intermediate data, even though we made clear to them that the code was provided so that THEY could choose which data to output.

      Having worked on one of the first open source cell phones I can tell you that Open Source is no panacea . People will still complain. Use SVN, they will demand GIT. If you have branches for development that may never see production.. people will demand that and demand that you fix bugs in things you never intend to bring to market. They will complain about code style, testing approaches… EVERYTHING.

      That said

      1. GISS code is available. It has even been refactored into Python.
      NO SKEPTIC RUINS IT.
      2. NOAA adjustment code has been available. NO SKEPTIC WILL RUN IT.
      3. CRU Code was made available. NO SKEPTIC WILL RUN IT.
      4. I created a R package that had

      A) CRU Methods
      B) Nick Stokes Method
      C) Tamino’s Method
      D) RomanM ( from climate audit ) Method.

      Result? NO SKEPTIC WOULD RUN IT.

      • Well, now that the problems have become obvious, maybe someone will fund a project and pay some objective people to run the open source project.

      • Objective people already have. You don’t like the objective answer. What you really want is for somebody who is not objective to run it. Then you can have what you want… the wrong answer.

      • > Use SVN, they will demand GIT.

        That makes them smart.

      • Steven, code is not always as well written as authors would like and at least in my experience does always reveal the conceptual complexities underlying the code.

        I’ll take your word for it that people who run your code get the same results as you do, but so what? I’ll take your word for it that your archived data is untampered by you, but so what? I’ll even take your word for it that you do not consciously seek predetermined results, but so what?

        No skeptic would run it? You use skeptic as a term of denigration. You can’t be a scientist without being sceptical.

        Strange days indeed!

      • You’re right, but why would I? I believe those are the results you get, other got much the same. But you all do the same basic steps, as that is now deemed SOP. I wanted to roll my own, I can assemble data, I wanted to see what was measured, not what “you guys” got out of some grinder.

        It was educational. You guys are basically wrong, got the science wrong, got the results wrong. Temps are going up, but you got the attribute wrong.
        So wrong I can’t help but wonder how it got so wrong?

      • > Temps are going up, but you got the attribute wrong.

        The Contrarian Matrix was all cranked up to Level 5 an now it gets back to Level 0.

        Just like that.

        And then people wonder why audits never end.

      • Just like that.
        And then people wonder why audits never end.

        But this is two different things, you have a temp series you have been arguing over for 20 years.
        Depending on where you want to pick whatever line you want you can find it. There’s a nearly 20 year pause, with a big el nino on each end, and they both show basically the same thing, surface temps are controlled by the oceans.
        So the game’s become prove ocean warming no matter what, because the atm follows dew point temperature, not co2. So if the oceans aren’t warming you have about 90% all natural oceans moving warm water around, nothing more.

      • > But this is two different things […]

        Of course they are.

        That’s the whole point.

        When one line is momentarily busted, you go for another one.

        Neverendingly.

        There are more than two kinds of lines.

        There are at least six.

        The ones at Level 0 is about lots of theories.

        The ones at Level 1 is about best practices.

        The ones on Level 2 are about lukewarm impacts.

        The ones on Level 3 are about doing no harm.

        The ones on Level 4 are about techno-optimis.

        The ones on Level 5 are about the truth that is out there.

        As you can see, Dr. John Bates backpedelled from a Level 5 accusations to level 1 concerns.

        You, OTOH, backpeddles to Level 0.

        See how the Contrarian Matrix works?

        You can thank GaryM for making me do it.

      • You, OTOH, backpeddles to Level 0.
        See how the Contrarian Matrix works?
        You can thank GaryM for making me do it.

        Hey I can’t help it the temp series doesn’t really matter, I invested 500 hours in it myself, maybe that was the price of entry.

        If you want to blame anyone blame the physics.

  50. The CCSP 1.1 report discussed the ocean and land data sets. In my role as Lead Author for one of the Chapters, I was tasked to assess what further evaluations were needed. But I was forced off the Committee when I did not conform to Tom Karl’s desired outcome (which clearly was to validate his data sets and parrot the conclusion he wanted. Here is what I wrote in my post

    https://pielkeclimatesci.wordpress.com/2011/11/30/further-documentation-of-inappropriate-behavoir-of-members-of-the-ccsp-1-1-committee-and-the-nrc-review-committee/

    “My view of that report was, and remains, that while there is useful and scientifically valuable information in it, it avoided discussing a number of substantive questions, including the robustness of the multi-decadal land surface temperature data of NCDC (Tom Karl), CRU (Phil Jones) and GISS (Jim Hansen).”

    Several relevant Climategate e-mails are included in that post.

    Further examples of the inappropriate dictates by Tom Karl, Tom Peterson, Peter Thorne and others are given in my posts

    https://pielkeclimatesci.wordpress.com/2011/11/29/conflict-of-interest-with-respect-to-an-nrc-review-panel-of-a-draft-of-the-ccsp-1-1-report/

    https://pielkeclimatesci.wordpress.com/2009/12/14/do-the-cru-e-mails-provide-further-documentation-of-a-conflict-of-interest-in-the-preparation-of-a-ccsp-climate-assessment-report/

    https://pielkeclimatesci.wordpress.com/2009/11/21/comment-on-the-post-enemies-caught-in-action-on-the-blackboard/

    https://pielkeclimatesci.wordpress.com/2009/02/23/another-failure-at-a-comprehensive-assessment-of-climate-the-revised-ccsp-report-by-karl-et-al-2009/ and other links embedded in that post

    as well as in my Public Statement

    Pielke Sr., Roger A., 2005: Public Comment on CCSP Report “Temperature Trends in the Lower Atmosphere: Steps for Understanding and Reconciling Differences”. 88 pp including appendices. http://pielkeclimatesci.files.wordpress.com/2009/09/nr-143.pdf

    This post and figure, presented at

    http://rankexploits.com/musings/2009/enemies-caught-in-action/

    exemplifies the type of ridicule that goes on behind one’s back at NCDC [in the figure presented by Tom Peterson].

    The post by John Bates, unfortunately is just a sample, based on my experience with that community.

    Roger Sr.

    • Mr. Pielke, I believe on your past blog that you had a post on Tom Karl peer reviewing his own work. Could you post a link to that if you have time.

      JD

      • The CCSP 1.1, of which he was Chair, is an example where he ruled over the assessment of his own data set. He shut me down when I asked questions about his methods, as I document in the links I have provided.

        A clear and obvious conflict of interest.

      • RP Thanks

    • Steven Mosher

      Post Code data and a CDR for all your work, otherwise we shall have to conclude that you have rushed your work for political purposes and it is not fit for use.

  51. Pingback: Has NOAA’s “Pausebuster” Been Busted?

  52. I have had several email exchanges with Tom Karl and his coauthors concerning the Karl (2015) paper and other related issues concerning the measurement of GMST trends, confidence intervals placed on those trends and appropriate time periods for comparing those trends, and all being prior to Karl’s retirement. We did not discuss the quality control placed on the older and newer versions that the Karl paper compared with regards to a slowdown in the warming rate in the most recent time period. I refer here to the older version as Old Karl and newer as New Karl.

    I suggested that for purposes of determining a warming slowdown that the years from 1975 to the year previous to any slowdown be compared to the period following up to the present time and not the 1950-2014 and 2000-2014 and 1998-2014 periods used in Karl and based on an IPCC comparison. I believe the authors agreed with me on this point.

    I also pointed out that using a linear regressed trend has to assume a linear trend relationship and, of course, one with no warming slowdown (or speedup) during the period being measured. The linear regressed trend for a shorter time period uses only the series information in the period of interest whereas other trend estimates, such as Singular Spectrum Analysis (SSA) and Empirical Mode Decomposition (EMD), do not assume a linear trend and use all the information in the series. I think the authors might have been partially convinced of this as they showed me a paper they were putting forth for publication that included an EMD analysis but continued to make comparisons with linear regressions. We had a discussion about EMD where I pointed out the need to keep the secular trend separate from the cyclical component and to determine whether these components were significant. It is at this point that our email exchange ended with no responses to my emails.

    On further literature searches and my own work I found that an improvement on EMD called Complete Ensemble Empirical Mode Decomposition (CEEMD) provided an excellent means of decomposing the GMST for not only observed GMST series but for CMIP5 model GMST series (SAT for land and SST for ocean) such that a reoccurring cyclical component could be removed and determined to be significant. The results of this analysis presents a very different picture than that derived from looking a linear regressed trends where the reoccurring cyclical component becomes either part of the noise or the trend.

    The links below shows the results of an CEEMD analysis of New and Old Karl GMST series and shows that (1) there is no significant difference in the later period trends between New and Old Karl GMST series and further (2) there is neither a slowdown or speed up in the trends for both New and Old Karl in shorter periods within the 1900-2016 period.

    This link shows the plots of Old and New Karl CEEMD secular trend and cyclical components where the residue is the secular trend and the imf 6 is the cyclical components:
    http://imagizer.imageshack.us/v2/1600x1200q90/922/Z7s2EY.png

    This link shows plots of the log(variance) versus log(period) that in turn shows that the trend and cyclical components are significant. The link also has a table showing the confidence intervals for trends over various time periods in the 1880-2016 period.
    http://imagizer.imageshack.us/v2/1600x1200q90/924/myylzB.png

    This link shows the residuals (original series minus the trend + cyclical) for the Old and New Karl series and the difference series of New minus Old Karl for the residuals, cyclical and trend components
    http://imagizer.imageshack.us/v2/1600x1200q90/922/Str6gQ.png

    I have emailed Zeke Hausfather with my findings and presented them on various luke warmer blogs with no apparent interest from anyone at this point.

    There are papers that have found and presented essentially the same results that I have here that can be readily found by googling. The key point is that there appears from these analyses a reoccurring 60- 70 year cyclical component that when accounted for in the secular trend presents a very different picture for the trend in the period 1880-2016.

    • Steven Mosher

      Code, data, and a CDR please. In machine readable format, per Bates.

      Because according to Bates and Curry if you don’t have a CDR then you have rushed things for political purposes.

      • At some point you pass the original objective and dive deeper into the murk of mindless drivel. Note to Steven – you are past the objective with this. We get it. Most likely on the first time you said it.

    • I think the only potential resolution to discussion of the differences of the ERSST v3b versus ERSST v4 versions and changes reported by Karl, but not done under his auspices, is to look at difference plots of the versions and attempt to understand how those differences might have been accomplished by a change in the adjustments.

      http://journals.ametsoc.org/doi/10.1175/JCLI-D-14-00006.1

      The other avenue for discussion of the validity of the changes is to understand how HadCRUT adjustments and use of input data differ from those reported in the link above. I believe both Boyin Huang and Zeke Hausfather in their papers point to this difference in results and methods and without any resolution.

      I do, however, think the much bigger issue in the observed GMST series is in determining the reoccurring cyclical component(s) to adjust the secular trend and at that point accounting for the resulting deterministic trend vis a vis increasing GHGs and other potential factors on global mean temperatures. Below I have listed links to the temperature series that I used in my CEEMD analysis and to other papers doing similar analyses using Empirical Mode Decomposition methods.

      Data for New and Old Karl provided by Tom Karl and coauthors for period 1880-2014:
      ftp://ftp.ncdc.noaa.gov/pub/data/scpub201506/

      Gridded data for ERSST v4 as referenced in Hausfather et al. (2016):
      https://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v4/netcdf/

      Gridded data for ERSST v3b as referenced in Hausfather et al. (2016):
      https://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/

      Link to NOAA Land, Ocean and Land/Ocean Global time series that includes ERSST v4 for Ocean
      https://www1.ncdc.noaa.gov/pub/data/noaaglobaltemp/operational/timeseries/

      Link to ERSST v3b and v4 data that with difference plot allows an update of V3b to 2016
      http://data.giss.nasa.gov/gistemp/updates_v3/ersst4vs3b/

      Links to some papers discussing analyses of Global Mean Surface Temperature series using Empirical Mode Decomposition methods:
      http://www.atmos.washington.edu/wallace/PDFs/2011%20-%20On%20the%20time-varying%20trend%20in%20global-mean%20surface%20temperature.pdf
      http://link.springer.com/article/10.1007/s00382-011-1128-8
      http://www.pnas.org/content/104/38/14889.full.pdf (probably the best description of the analysis that I have reported here)
      http://engine.scichina.com/publisher/scp/journal/SCES/60/2/10.1007/s11430-015-5465-y?slug=full%20text

      • Ken,

        Thanks you for an excellent comment. Please see my question to you in my response to Geoff Sherrington’s comment below.

    • Geoff Sherrington

      Ken,
      Thank you for this work. Even generalists like me have long winded what lies further down than rudimentary least squares linear analysis. It is telling that you are an outsider reporting a lack of interest by those who should be at the vanguard.
      There have been various blogged opinions about whether (using dodgy labels) old Karl is different to new Karl. Can I please suggest that you summarise your findings here as crisp sentences designed to be read as the final word on the difference?
      The overall topic on various blogs is fragmented, with some fragments now at the stage (maybe) of putting to bed. The more, the sooner, the better.
      Thank you again. Geoff

      • Geoff Sherrington

        Wondered. Time to abandon anticipatory verbiage via tablet keypad.
        Geoff

      • Geoff, thanks for your singular reply to my posts here. I prefer not to pontificate or get into personalities in discussing these matters but since you have implied that you really want to know my thinking on this issue I’ll attempt to summarize it here.

        1. I really find the pontifications and back and forth on personal issues on who said what from all sides of AGW issue terribly tiresome and a waste of the bandwidth here at Climate Etc that Judith Curry has provided here for discussion of climate science issues.

        2. I think that a number of climate scientists on all sides of the AGW issues tend to report and publish evidence and analyses that favor a preordained stand on these issues. It is as though we have adversarial lawyers from opposite sides of issue presenting their arguments and in order for the readers of these documents to make a judgment requires hearing both sides or when one side is absent doing their own analyses. While I am aware of this process being the case in most branches of science, whether admitted or not, with climate science there is that added flavor of the political influences.

        3. I have no reason to believe that Karl and his coauthors or those making the adjustments from ERSST v3b to v4 went beyond the tendency to report evidence and analyses favorable to a given position and as such the emphasis should be on analyzing the methods used and any lack of sensitivity testing that could well show what was ignored or overlooked by the authors.

        4. I have presented methods using CEEMD here that have been used elsewhere in analyzing global mean surface temperature series (GMST) but have not gotten the attention in my opinion that they deserve from the climate science community. Finding a method to decompose the temperature series and looking for reoccurring cyclical components that could confound the secular trend estimations is something that in my judgment should be of utmost importance.

        5. These methods using Empirical Mode Decomposition paint a picture of a reoccurring cyclical component adding to the modern warming rate and when removed producing a secular trend that shows gradually increasing temperatures from 1900 to present time without a significant increase expected given the rapidly increasing levels of GHGs in the atmosphere from approximately 1975 to present time.

        6. Karl and his coauthors did not appear interested in the separation of the cyclical component from Empirical Mode Decomposition into a secular trend and cyclical components and further showing that those components were statistically significant but rather wanted to combine those components which paints a very different picture as I have noted above. I have pointed out my analysis to Zeke Hausfather in an email and received no response. It is these failures to respond to new ways of analyzing these temperature series, and other aspects of climate science as well, that bothers me more about the current state of climate science than any deviousness that might be attributed.

      • Ken, plus many.

      • Geoff Sherrington

        Ken,
        Mea culpa.
        I meant mainly to suggest you wrap up your statistical work with a clear, short summary that could put an end to the “Is, Is not” claims of differences between these time series.
        Thank you for the longer discourse. The important part of this matter is the quality of the data that comes through the kerfuffle for the permanent record.
        In that regard, I note that Dr Curry has succeeded with a blog of enviable reputation through, inter alia, wise and pertinent choice of material. I admire her comparative silence when she must know more of this story than those like me who seek to speculate from a lower knowledge base. Poker face?
        Geoff

      • Geoff says:

        It is telling that you are an outsider reporting a lack of interest by those who should be at the vanguard.

        Four other examples are:

        1. Lack of interest in outsider Steve Mcintyre’s expose of the errors in the Michael Mann’s Hockey stick analyses and subsequent defence and denial of it.

        2. Initial lack of interest in outsider Nic Lewis’s much lower estimates of ECS than those produced from GCM analyses, followed by denial.

        3. Lack of interest in the lack of valid justification for the CAGW meme i.e. that GHG emissions are dangerous or net harmful:

        – The evidence I presented in the 12 comments starting here https://judithcurry.com/2016/11/25/week-in-review-science-and-policy-edition-3/#comment-826494 was ignored.

        – Scotese’s canalysis that GMST averaged 7 C warmer than now for the past 650 Ma – since complex life began was ignored. (Life thrived for most of this period).

        – I asked on Real Climate web site, and asked Gavin Schmidt by email, if there is a better (i.e. more widely accepted) chart of GMST for the Phanerozoic Eon (past 542 Ma) than Scotese 2016 chart (Figure 15 here: https://www.researchgate.net/publication/275277369_Some_Thoughts_on_Global_Climate_Change_The_Transition_for_Icehouse_to_Hothouse_Conditions ). My questions were ignored (or dismissed without answering the question).

        – I pointed out here: https://judithcurry.com/2017/01/29/the-threat-of-climate-change/#comment-836115 that Scotese’s charts (Figures 12, 13, 15) suggest GCMs may be grossly exaggerating warming in the tropics and underestimating warming in mid and high latitudes and the IAMs therefore may be grossly exaggerating damages and underestimating benefits of GW and of increasing GHG concentrations in the atmosphere. This critical point for justification of ther CAGW meme was largely ignored.

      • Ken,

        This seems like the takeaway message to me:

        5. These methods using Empirical Mode Decomposition paint a picture of a reoccurring cyclical component adding to the modern warming rate and when removed producing a secular trend that shows gradually increasing temperatures from 1900 to present time without a significant increase expected given the rapidly increasing levels of GHGs in the atmosphere from approximately 1975 to present time.

        In short,

        – most of the warming 1975 is due to natural causes, not human caused GHG emissions.

        – GMST response to increasing GHG concentrations in the atmosphere is much less than GCMs predict.

        Is that a reasonable conclusion to draw from your comment?

    • Very well written comments Ken (and Geoff). Thank you.

      • The persistent use of simple linear regression with a diagnostics and a transparency discussion on limitations conveys a sense of convenience and laziness and for me throws up red flags all over the place. I do not so much doubt a ‘slope’ as worry over possible less than comprehensive diligence in its quantification.

      • …without diagnostics and a transparent discussion…

  53. Given the inadequacy of the collection methods involved, after we homogenize all the data we probably should also pasteurize it to prevent inflammatory responses.

    • Yes! I insist that we do.
      Can we have a free-range, organic, fair trade, rainforest alliance certified, gluten free, lactose free, low fat, low sugar, high fibre, low GI, no nuts, no artificial flavors, no artificial colors, kosher and hallal version too?
      Please – for, like, the kids future, you know?

      You can laugh at the above (you should, BTW), but I have seen “free-range organic honey” on the shelves – no joke.

  54. Evenin’ all,

    It’s well past bedtime over here in the once Great Britain. I haven’t even got onto David Rose’s verbiage yet, and my long standing query about Mr. Rose’s imagery has still not been addressed as far as I can ascertain.

    What with one thing and another I’ll think I’ll try a different tack. For supper here’s a remarkably prescient video from the glorious Dan & Dan which reveals to the world exactly how a large proportion of we Brits view the Mail’s award winning investigative “churnalism”:

    https://youtu.be/5eBT6OSr1TI

    See if you can spot the reference to Climategate.

  55. Pingback: US govt agency manipulated data to exaggerate climate change – whistleblower

  56. And finally, thanks to a variety of intriguing coincidences on Twitter throughout today, an ad hoc team of celebrity international chefs have concocted this warming and educational nightcap for your delectation:

    https://twitter.com/jim_hunt/status/828756200225439744

    You have to keep clicking through to the very bottom of the virtual mug in order to experience the full benefit.

    I look forward to seeing you all again in the morning (UTC). Thank you and good night from May or May not land.

  57. Another perspective. The data that killed the hiatus was ERSSTv4 which was not Karl’s doing. Karl only wanted to publish his paper to explain it as that data came out. Bates’ complaint there was that the hiatus-killing ERSSTv4 data was delayed in coming out by Karl. So here we see Bates in favor of getting ERSSTv4 out even quicker than Karl. Puts a twist on things, doesn’t it.

    • This perspective puts lie to those headlines in the Daily Mail and House press release too. The whistleblower may have had something against Karl, but not with the all-important data adjustment that was the main result of the paper, which he actually wanted published quicker. What would Rose and Smith think if they knew that?

  58. “The procedures developed by John Bates are absolutely essential for certifying these datasets, as well as their uncertainties, for a regulatory environment.”

    Just so.
    People who work in this area have 3 reasons to follow such policy:
    1) the people paying the money have asked you to – you should do what you are paid to do;
    2) tracability and replicability are enhanced – others can check your work (or you theirs) more easily;
    3) All the arguments in the previous thread re: what is the “correct” answer can be ignored – you performed the work according the the relevant published standard. If this standard is somehow “wrong”, “incomplete”, “inapplicable” etc, the person making such a claim has an argument with the standard, not your work – no more “he said/she said” on a per-paper basis. If the policy is wrong, argue the policy is wrong and get it changed! This not only “fixes” YOUR results, it fixes other peoples as well, and puts ALL work on the same footing.

    Is it painful? Certainly.
    Are there funds to do it? If not, your grant proposal should include the costs of doing the required work.

    Dodge it if you wish – just don’t expect anyone to take you seriously if and when you do.

  59. Steven Mosher

    Judith

    Do you think it is relevant that Karl demoted Bates years ago

    https://arstechnica.com/science/2017/02/article-names-whistleblower-who-told-congress-that-noaa-manipulated-data/

    Ars talked with Thomas Peterson, a co-author on the Karl paper who has since retired. Peterson provided some useful context for understanding Bates’ allegations. The satellites that Bates worked with were expensive hardware that couldn’t be fixed if anything went wrong after they were launched. The engineering of the software running those satellites sensibly involved testing and re-testing and re-testing again to ensure no surprises would pop up once it was too late.Bates expected the same approach from his surface temperature counterparts, but Peterson explained that their work with weather station data was not nearly so high-stakes—problems could easily be fixed on the fly. The engineering-style process NOAA was using for endlessly double-checking the software for all dataset updates could drag on for quite a long time—years, in fact—and Bates opposed any attempt to speed this up. Peterson and other scientists were naturally anxious to incorporate changes they knew were scientifically important.”

    “Bates also claims that there were bugs in the land station database software that were ignored in the Karl paper. But according to Peterson, the slight day-to-day variability seen in the software’s output was simply the result of the fact that new data was added every day. Stations that straddled statistical cut-offs might fall on one side of the dividing line today, and on the other side tomorrow. There was nothing wrong with the software, they realized. It was just silly to re-run it every single day.”

    “There may also be something beyond simple “engineers vs. scientists” tension behind Bates’ decision to go public with his allegations. Two former NOAA staffers confirmed to Ars that Tom Karl essentially demoted John Bates in 2012, when Karl was Director of NOAA’s National Centers for Environmental Information. Bates had held the title of Supervisory Meteorologist and Chief of the Remote Sensing Applications Division, but Karl removed him from that position partly due to a failure to maintain professionalism with colleagues, assigning him to a position in which he would no longer supervise other staff. It was apparently no secret that the demotion did not sit well with Bates.”

    • Steven,

      You might find of interest the discussion upthread between Nick Stokes and me about the Ars Technica article. Also, note the article’s weird title.

      • Steven Mosher

        I find bates recanting more interesting.

        Do you think Rose and Judith Owe Karl and Thorne an apology

        The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

        Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

        However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

        “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  60. Judith ==> I find the “taking sides” interesting and, possibly, worrisome.

    There seems to be a number of denizens with vested interests in the Karl/Petersen/Huang methodologies who are reacting to the Bates revelations as if Bates was somehow a threat to them personally.

    Based on my early training in [mostly commercial] intelligence work, this extreme defensiveness and efforts to rapidly shut down [discredit] the source of the revelations indicates that there is a lot more technically dubious information (or untoward scientific behavior) to come out yet — either from Bates or others at some of the alphabet agencies and departments.

    Time will tell. And, like today’s national politics, it is guaranteed to the a continuing round of tumult. We live in interesting times.

    • And my experience is people who say what you just said say it because they’ve got nothing and are feeling like they ain’t gonna ever get anything.

      It’s keeping your last-ditch hope alive.

      People are upset because this is a low down dirty rotten below the belt stunt.

      • richardswarthout

        JCH

        You, Mosher, Stokes, Etc. have entered the world of disgusting; believing anything that is anti-Bates, turning off your B.S. meter. Ex. Titley’s unbelievable story after being at NOAA for only 12 months 2012-13; if confronted by such a story from the witness stand, any good lawyer would decimate him.

        Richard

    • ‘There seems to be a number of denizens
      with vested interests in the Karl/Petersen/
      Huang methodologies who are reacting
      to the Bates revelations as if Bates
      was somehow a threat
      to them personally.’
      ‘Climate Audit ‘n
      Climategate,
      deja vu – all
      over again!

      • Steven Mosher

        You missed this

        ‘The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

        Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

        However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

        “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

    • Kip, I just guest posted another example over at WUWT. Went there because he had already picked up and reposted Paul Matthews post proving that Bates was right about adjustment software instability.

      • rud ==> It will take a whole village of whistle blowers to shake all the snakes out of the rafters at NOAA (if there be any — which I believe there are).

      • You two are so lost Siri is speechless.

      • It’s worse than we thought, JCH:

        I later learned that the computer used to process the software had suffered a complete failure, leading to a tongue-in-cheek joke by some who had worked on it that the failure was deliberate to ensure the result could never be replicated.

        Karl did not only put his right thumb on a program that dynamically statistical values, he put his left thumb on it so it breaks at will to hide Da Paws.

        Let’s Make INTEGRITY ™ Great Again!

    • Kip, quoting Abraham Lincoln would also be appropriate here: “You can fool ….”

      The only thing that really matters is those insane policies based on IPCC climate model nonsense. We deplorables have Trumped them, finally.

      Eat your hearts out, you climate alarmists, snowflakes, social justice warriors, socialists, one worlders and totalitarians of all stripes. The money spigot is being closed.

  61. Another volley in the debate: “DO NOT BUY THE HOUSE SCIENCE COMMITTEE’S CLAIM THAT SCIENTISTS FAKED DATA UNTIL YOU READ THIS” by Kendra Pierre-Louis at Popular Science — “NO CREDIBLE EVIDENCE SUPPORTS THAT NOAA FABRICATED DATA; EVIDENCE STILL POINTS TO CLIMATE CHANGE.”

    It opens with;  “{Karl 2015} debunked the notion of a climate “hiatus” or “cooling.” This paper in Science provided evidence for one of the theories explaining the “pause” or “hiatus”, about which climate scientists have published many scores of papers. The authors did not disrespect the work of their peers by saying they debunked it.

    The author quotes two climate scientists who go to the heart of Bates’ article at Climate Etc. Zeke Hausfather of Berkeley Earth confirms one of Bates’ key claims, a significant violation of NOAA policies.

    “His primary complaint seems to be that when researchers at NOAA published this paper in Science, while they used a fully developed and vetted ocean temperature product, they used an experimental land temperature product. …Because climate data comes from a number of different sources, methods of handling that data go through a vetting process that ultimately dictates the use of one for the official government temperature product. That can mean controlling for known defects in the devices that gather climate data or figuring out the best way to put them together. The product that Karl used for land temperature data hadn’t finished that process.”

    What is the defense? Some people believe that NOAA’s policies were subordinate to their personal opinions. Hausfather said “That said, the land temperature data they used in the paper is certainly up to the standards of an experimental or research product.” Kevin Trenberth of NOAA said: “Bates was complaining that not all of the data sets were being done as thoroughly as he wanted to,”

    The global surface temperature dataset is one of NOAA’s flagship products, for obvious reasons. NOAA has policies to protect the process and the resulting dataset. Bates spoke out for that process. Eric Davidson, President of the AGU, said that the significance of this issue will be reviewed and assessed by climate scientists. Let’s wait for their verdict. But journalist Kendra Pierre-Louis believes that instead we should listen to her:

    “So what does that mean for those of us on the outside? Not much.”

    • “The global surface temperature dataset is one of NOAA’s flagship products, for obvious reasons.”
      Yes. And Karl’s paper did not release a dataset. His paper was titled “Possible artefacts of data biases in the recent global surface warming hiatus”. It was a research paper that identified biases. He made it clear that they were in fact using the ISTI dataset (as published, not a NOAA product), which was on the way to becoming GHCN V4, but he didn’t claim to be using GHCN V4. He said

      https://s3-us-west-1.amazonaws.com/www.moyhu.org/2017/02/k15.png

      • This relates to some important points I noted above.
        1. It was ERSSTv4 that killed the hiatus.
        2. Karl merely used ERSSTv4 and not was even a co-author for that dataset.
        3. Bates fully accepted ERSSTv4 and even wanted it released before Karl published his paper.
        4. Karl’s paper described the effect that the new data had on the hiatus. He was just the messenger.
        5. Bates’ quibbles related to the land data in the Karl paper that did not have much effect on the hiatus by itself.
        David Rose and Lamar Smith completely got the wrong end of the stick on this. They shot the messenger when the damage was already done by ERSSTv4.

      • It is inconvenient to the skeptics that Bates accepted ERSSTv4 and wanted it published more quickly than Karl. They need to find someone else, clearly not Bates, if they want to relitigate that part.

      • Nick,

        How sad that none of the climate scientists cited in these articles — such as Trenberth and Hausfather — thought to give that response. In fact their responses to Bates’ discussion of the temp datasets were quite different than yours.

        I’ll take the advice of the AGU president and let this play out among them, watching their answers.

      • “The global surface temperature dataset is one of NOAA’s flagship products, for obvious reasons”

        you have to LAUGH-

        WMO- “Because the data with respect to in-situ surface
        air temperature across Africa is sparse, a one-
        year regional assessment for Africa could not
        be based on any of the three standard global
        surface air temperature data sets from NOAA-
        NCDC, NASA-GISS or HadCRUT4 Instead, the
        combination of the Global Historical Climatology
        Network and the Climate Anomaly Monitoring
        System (CAMS GHCN) by NOAA’s Earth System
        Research Laboratory was used to estimate
        surface air temperature patterns”

        Some NOAA Flagship!!!! it’s not even there.

      • Picotech makers of high precision temp gauges-
        Conclusion
        “High precision temperature measurement is possible through the use of well-specified and suitably calibrated sensors and instrumentation. However, the accuracy of these measurements will be meaningless unless the equipment and sensors are used correctly”

        NOAA cut out the middleman dealing with measurements that could be “meaningless” and opted for estimations instead in Africa.- You can’t go wrong like that- and all measured to tenths of a degree.

      • What really annoying is that, Picotech, makers of high precision temp gauges cannot get within one degree of accuracy in a controlled environment and you all argue over tenths of a degree when half the world has no temp data-

        “Consider what you are trying to measure the temperature of. An example that seems simple at first is measuring room temperature to 1°C accuracy. The problem here is that room temperature is not one temperature but many.

        Figure 1 shows sensors at three different heights record the temperatures in one of Pico Technology’s storerooms. The sensor readings differ by at least 1°C so clearly, no matter how accurate the individual sensors, we will never be able to measure room temperature to 1°C accuracy”

  62. “The fundamental issue is this: data standards that are ‘ok’ for curiosity driven research are NOT ‘ok’ for high impact research of relevance to a regulatory environment.”

    Bravo.

    Nor is scuttling the computer before retirement so the garbage cannot be reproduced in effigy. “We all go to heaven on a hard disk drive”…Rodney Crowell.

    • Steven Mosher

      Except when Judith testifies before Congress and uses data from a Beta version of UAH… no CDR.. No Process..

      One set of rules for karl
      Another Set of Rules for Judith and Pielke

      • Mr. Mosher. Pop your head above the Weeds.

        Karl is one item being used by the political types in efforts to fundamentally change our societies, economies, and energy systems.

      • Thanks for responding Steven. Don’t know all of the particulars of the data sets used by Judith and Roger. I genarally agree with you and Nick that some adjustments are legitimate.

        Some are not.

        No dialogue, no progress.

  63. Pingback: manipulating an agenda and losing … | pindanpost

  64. Pingback: Ap vil fjerne flypassasjeravgiften og innføre en annen avgift a.la. drivstoffavgift – Venner av Rygge

  65. The Pacific Ocean Climate shift is alive and well – and data manipulation doesn’t change this. And the next shift is due within a decade.

    I have been through this before The 1976/77 shift foretold the coming – drawn there by greenhouse gases – of endless El Nino. Well no – I thought – it has happened before. This is what I had to work with.

    http://hydrologie.org/redbooks/a168/iahs_168_0327.pdf

    It looks simple but it is inspired science based on decades of observation. We made tentative steps towards a broader analysis and stratified stochastic water resource analysis, Then it came down on Robin Warner’s head – stumbling in mid-sentence as he realised the impossibility of reconciling this with the demands of global warming doofus’.

    Hydrology and water resource management were set back 50 years at least. Whole schools arose dedicated to losing any continuity in hydrological knowledge and pretending that the world began in 1950. Has there been such an era of incomprehensible madness in science?

    But nature has the last laugh.

    https://www.esrl.noaa.gov/psd/enso/mei/ts.gif

    A continuation of the ENSO pattern established since the 1998 shift will persist until the next shift – within a decade. I have an idea that it is solar modulated.- and the next shift is to yet cooler. It is not a warmer – cooler – warmer pattern. But you don’t know that because your world started in 1950.

    https://watertechbyrie.com/2017/01/12/an-earnest-discovery-of-climate-causality/

    I picked it last time – why not now? I spoke about it 2007 when it was somewhat clearer and the IPCC had utterly failed. I was immediately listed on the SourceWatch hit list. We know a lot more about it now. And feel free to pull any nonsense out of your arse in rebuttal.

    https://watertechbyrie.com/2017/02/06/2141/

  66. In December 2015, Judith Curry presented testimony to the Senate Subcommittee on Space, Science, and Competitiveness, in a hearing timed to coincide with the Paris climate summit.

    In her testimony, which is clearly intended to influence policy, Curry includes a plot of UAH lower tropospheric temperatures. Now, no number was provided to indicate the version of UAH data shown, which is itself tantamount to misconduct as we’re learning here. However, by overlaying different versions it’s clear what’s shown is some variant of UAHv6. This is a dataset which has not even been peer reviewed (still to this date), let alone any of the other checks being talked about here. How does that fit in with the policy-relevant data integrity issues being discussed here?

    • UAH v6 lines up nicely with RSS, and UAH has a new method to go along with new software. Im sure Dr. Curry is shaking in her boots over this revelation. But keep trying

      • Read the top of this post. This is about process. UAHv6 being ‘close enough’ to other (sort of) independent analyses is besides the point.

        Are you saying you have no problem with the use of non-peer-reviewed. non-operational datasets within a regulatory environment?

      • Are you serious? Listen if Mears comes forward and craps all over UAH then we will all take notice. His opinion matters. And if you can demonstrate that something is amiss with V6, please present to the class. Let the peer review begin.

      • Paulski, many important improvements were made to the V6 product. Read up on them

      • No, paulski0; you use it to forestall hasty policy making on issues that fundamentally affect our society, economy and energy systems.

      • I’m not saying anything about whether or not there any problems with v6. I’m talking about proper process. Have you read the top of this article yet? That’s what this is supposed to be about.

        And you haven’t answered the question: Are you saying you have no problem with the use of non-peer-reviewed. non-operational datasets within a regulatory environment?

      • Paulski, I think RSS would have been the better choice for JC. But because she used a dataset which virtually mirrored RSS, i think its fine. I like your point, ans i like that you are demanding for standards. Im not sure JC did anything wrong here. And again, unless Mears objects, which he likely wont…

      • “Listen if Mears comes forward and craps all over UAH then we will all take notice. His opinion matters. “

        OK, Here it is:

      • Nick, thank you for the link. I watched it, im sure you didnt because Mears does not slam UAH here. He comment on an earlier version. And if you have kept up, Mears and Christy have savagely improved the others datasets over time.

    • @paulski0
      Do you not think it would be a good idea for there to be some kind of quality check on data prior to it being presented within a regulatory environment? For that to happen, a policy regarding the presented data would need to be in place. Neither Judith Curry nor owenvsthegenius have any business setting government policy. That would be the responsibility of a government agency such as the NOAA.

      • @paulski0
        Do you not think it would be a good idea for there to be some kind of quality check on data prior to it being presented within a regulatory environment? For that to happen, a policy regarding the presented data would need to be in place. Neither Judith Curry nor owenvsthegenius have any business setting government policy. That would be the responsibility of a government agency such as the NOAA.

        lol, ironically as a minimum this is exactly what Karl did. He didn’t follow NOAA/NCDC procedures.

      • Peer review is a pretty well-accepted quality check. Beyond that, it’s possible something like Bates’ data quality matrix could be useful to grade things, but I don’t think it’s as clear-cut as Bates believes.

        I suspect the best approach is to have a review body of experts which assesses the scientific literature and produces a report, say every 6 years, for policymakers. They take into account matters of data quality among the many other issues to consider.

      • Peer review has its place but it is not nearly rigorous enough to ensure data quality for important surface temperature databases such as those provided by a government agency. Maybe John Bates’ CDR methodology is overkill, but if the goal is to end up with a reliable database then a reasonably rigorous QA process is needed, first for compiling temperature observations and then for applying homogeneity and spatial averaging algorithms to them.

    • Simply put it does not. It is a false equivalence. On the one hand we are talking about a government funded entity following or not following its own policies. The jury may still be out on this. Nothing so far is very convincing one way or the other as far as I can determine. I am not convinced that the rules were for a very particular set of circumstances and not applicable to the report in question. However, scientists that are employed by the entity need to follow the entity’s rules. As far as I can tell this is the issue, and the allegation should be thoroughly investigated. If it can be found that the scientist(s) in question did not follow the rules then there should be consequences. Note that in this case the onus is on the ones bringing the charges to put forward evidence and “prove” their case. It is never, in my mind, the job of the person being charged to prove their innocence.

      On the other hand we are talking about a scientist that worked for a University giving testimony before congress. The scientist would be only restricted by the norms of the profession, the norms of the university, rules of the committee, and the truth as far as the scientist understood it. How could the rules of a government entity that the scientist is not employed by have any jurisdiction here? Your whole argument makes no sense to me.

      • I was thinking that testimony might also be restricted by congressional policies governing quality of evidence such as climate data (if such policies existed). If policies don’t exist, then I agree with your statement that the presenting scientist should testify as honestly as possible bound by the ethics of their profession and the policies of their employer.

        However, if a congressional policy governing the quality of climate data were to be put in place, I would expect congress to go to a government agency like the NOAA to help formulate it.

      • willbo1,
        My comment was not aimed at you, but paulskio above. I doubt my comment makes a lot of sense to your comment.

      • Internal NOAA policy is one thing, but the claim has also been made that Karl et al. misled Science magazine and reviewers, the science world and policymakers. Which can be the only reason for the wide-ranging investigations Judith is encouraging. There’s no reason for independent investigations into whether or not an internal policy guideline was breached – it’s simply a matter for NOAA to decide.

        Put it this way: Imagine Karl and co-authors were not NOAA employees and instead independent researchers using the same publicly available datasets and software algorithms to produce exactly the same paper. In this scenario there is no question of breaching internal NOAA policy. Have they done anything wrong? If not, there is no science ethics issue here.

      • > Imagine Karl and co-authors were not NOAA employees and instead independent researchers using the same publicly available datasets and software algorithms to produce exactly the same paper. In this scenario there is no question of breaching internal NOAA policy. Have they done anything wrong? If not, there is no science ethics issue here.

        Come on, PaulS.

        That’s a thought experiment. It makes you build a conceptual modulz. We all know that modulz are stoopid. That’s why we call them stoopid modulz.

        Something wrong has been done. We have a whistle blower. He says something wrong has been done.

        What more do you want? Evidence? We have been told there was mounting evidence.

        That’s enough for me.

        We need some data extreme vetting to make sure that we put disclaimers at the bottom of each and every page of every paper that NOAA will ever produce.

        That’s the only way Western Civilization will survive this crisis.

        Thank you.

  67. Pingback: Los alarmistas admiten el guarreo de la NOAA con la temperatura global | PlazaMoyua.com

  68. Sure – it is about as massively tweaked as any of the surface records.

    http://www.drroyspencer.com/wp-content/uploads/V6-vs-v5.6-LT-1979-Mar2015.gif

    And the elephant in the surface record is?

    https://i2.wp.com/www.nature.com/nature/journal/v491/n7424/images/nature11575-f1.2.jpg?zoom=2

    They are very different beasts. Can this explain the really quite immaterial difference or do you just want to run with the urban doofus hipster meme factory product?

    http://woodfortrees.org/graph/hadcrut4gl/from:1980/plot/rss

    Interesting as all this is… OMG.

    Here’s my solution to the ‘climate problem’ – what’s yours?

    https://watertechbyrie.com/2017/02/06/the-climate-problem-and-the-solution/

    • “Sure – it is about as massively tweaked as any of the surface records.”

      Robert:

      No comparison my friend.
      All the surface date is available in it’s raw and adjusted form.
      The adjusted data make no sig difference to the warming of the 80 yrs worth of trend.
      In fact the warming most noticeable was that done for the ocean record prior to that and that actually reduces the rate.

      http://cdn.arstechnica.net/wp-content/uploads/2016/01/noaa_world_rawadj_annual.png

      The satellite record comes from just one operating sensor, which has to be assumed to be calibrated correctly.
      UAH have said that they assume it is the current AMSU.
      RSS say ( honestly and sensibly ) that they do not know which is wrong,the MSU prior to ’97 or the AMSU since.
      I have posted evidence that it is running increasingly cold with respect to (multiple) radiosondes comprising the RATPAC A series..
      Carl Mears of RSS says thus:

      ” A similar, but stronger case can be made using surface temperature datasets, which I consider to be more reliable than satellite datasets (they certainly agree with each other better than the various satellite datasets do!).”

      RSS are now recommending using v4.0 TTT instead and have ditched (as in not updating) the TLT version in v3.3.
      UAH are currently on v6.05.
      Both are reliant on complex algorithms to give a brightness temp for a depth of the troposphere, and need to know local noon under the orbit in order to correctly assess a max temp. This process is aided by a climatological GCM “fix”.
      SO yes, of course we all know that the Sat Trop temp data series are better and have fewer “adjustments” that are all documented and presented as raw brightness numbers with the code and GCM adjustements made available.
      (at least in UAH’s case)
      I mean it just stands to reason.

      I refer you back to the graphs (this FI)…….
      http://postmyimage.com/img2/995_Tropospheretrends.png

      of the increasing cool bias displayed by the current AMSU sensor and ask that if the surface temp data relied on just ONE INSTRUMENT (not 1000’s) and that it were drifting warm to multiple satellite sensors (not just 1) … then you would be happy?
      I mean – it just stands to reason that you would. (sarc)

    • And, Robert, satellite data cover only the period leading up to the peak of a 60 to 70 year climate cycle. The 37-year trend is just over 1 degree C, ending on a super El Nino. Scary? Not.

      It is a fact that the best ESTIMATE indicates an increase of less than 1 degree C over the instrumental period. Scary? Not.

      The recent deceleration in the warming trend is so obvious that even IPCC AR5 had to use “expert opinion” to reduce climate model near-term projections. Scary? Not.

  69. richardscourtney

    Dr Curry:

    In your above article you write,

    ” Most journalists and people outside of the community of establishment climate scientists ‘get it’ that this is about the process of establishing credibility for climate data sets and how NOAA NCDC/NCEI have failed to follow NOAA’s own policies and guidelines, not to mention the guidelines established by Science for publications.”

    Well, if it is about that then it misses the main point.
    Adherence to correct procedures for doing something wrong does not make what is done right.

    All – yes, all – time series of global and hemispheric temperatures fail to be scientifically valid.
    I explain this in your other thread at
    https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-836664
    and
    https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-837095
    As I there explain repeatedly, the fundamental issue is that

    All global temperature data will continue to be bunkum until
    (a) there is an agreed definition of global temperature that all adopt and none frequently change
    and
    (b) there is some possibility of an independent calibration standard for global temperature.

    Until then all global and hemispheric temperature data remains less scientifically valid than phrenology. Paymasters say what they want to be be suggested and the compilers of global and hemispheric temperature data can and do provide it.

    That is true, and until it is corrected then any discussion of procedures for producing the junk which is asserted to be global temperature time series is akin to a discussion of how many angels can dance on the head of a pin when what is said to be a pin and what is said to be an angel change almost every month.

    Richard

    • Steven Mosher

      Too bates recanted

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  70. Defenders of NOAA’s scientific integrity might want to look at the under reported expose of NOAA management’s hyping of ‘ocean acidification’ impacts over the objections of their own expert scientists.

    NOAA planted a fake scare story written by its PR people in the NYT for the sole purpose of raising its profile.

    http://junkscience.com/2015/12/exclusive-ocean-acidification-not-a-current-problem-top-noaa-scientist-insists-in-foia-ed-e-mails/

    • Those changes in protocols referred to – oyster producers in WA ship their larvae to Hawaii to for the first few weeks of their development cycle. At a certain point in their growth they are basically immune to any effects in pH change.

      Now one has to wonder – if OA is a direct result of increasing atmospheric CO2, why would Hawaiian waters be less acidic than Washington waters? Might it have something to do with WA and OR having significant upwelling of deep ocean currents? Can scientists distinguish between a CO2 molecule originating in the atmosphere and one from the cold ocean depths?

    • Judith

      Well, it might be brilliant but how do we access it without signing up?

      Mind you I have always thought this was about not following protocol rather than fraud so have been reading the outraged comments from both sides in bemusement

      Tonyb

      • Also here without needing sign-up (I think it’s the same story, by eenews anyhow):
        http://www.eenews.net/stories/1060049630

        P.S. Judith I just sent a submission to what I hope is your new email address.

      • Thanks for the new links. In have read the article available without subscribing. I can’t see what the fuss is about as surely bates is not accusing anyone of anything more serious than in not following the protocol of publishing?

        In my opinion his closing remarks were more interesting

        “You really have to provide the most objective view and let the policymakers decide from their role,” Bates said. “I’m getting much more wary of scientists growing into too much advocacy. I think there is certainly a role there, and yet people have to really examine themselves for their own bias and be careful about that.”

        With the advent of twitter the advocacy and beliefs of scientists are being exposed in their tweets and interaction with those of a like mind

        Tonyb

      • > I can’t see what the fuss is about as surely bates is not accusing anyone of anything more serious than in not following the protocol of publishing?

        It surely depends upon what “thumb on the scale” means, TonyB.

        Unless that’s “verbiage” too?

      • tony –

        =={ I can’t see what the fuss is about as surely bates is not accusing anyone of anything more serious than in not following the protocol of publishing? }==

        From your boy, Rose:

        –snip–
        Dr Bates said: ‘They had good data from buoys. And they threw it out and “corrected” it by using the bad data from ships. You never change good data to agree with bad, but that’s what they did – so as to make it look as if the sea was warmer.’
        –snip-

        So, Bates accusing them of throwing out data (no doubt, holding it with the other fingers when doing so since their thumbs were “on the scale”) is no more serious than not following protocol?

      • Willard

        As far as I can see the ee interview is new and separate. In the link you highlight from ce bates appears to say:

        “So, in every aspect of the preparation and release of the datasets leading into K15, we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation. I finally decided to document what I had found using the climate data record maturity matrix approach. I did this and sent my concerns to the NCEI Science Council in early February 2016 and asked to be added to the agenda of an upcoming meeting. I was asked to turn my concerns into a more general presentation on requirements for publishing and archiving.”

        The ee article seems altogether more conciliatory whereby bates appears to be saying the thumbs on the scale relate to work on the climate hiatus rather than Igniting a general scandal that the figures Karl is involved with are fraudulent, which is what sceptics have seized on.

        They appear slightly contradictory . However as our official and highly paid auditor I will listen to your more learned interpretations as to what is going on here.

        Tonyb

      • > As far as I can see the ee interview is new and separate.

        I knew I could count on you to extend the limits of justified disingenuousness, TonyB.

        Teh Donald adds new and separate tweets each and single day. That does not justify their frequent inconsistencies, doesn’t it?

        Until Dr. Bates takes back his “tip on the scale” comments and David Rose amends his text to reflect his sudden change of heart, I’m afraid he’ll have to own it.

        And Judy will have to own the fact that the “verbiage” doesn’t end in David Rose’s text.

      • Willard

        There appear to be inconsistencies between the use of the thumbs on the scale quotes as they appear to have a different context in each of the articles. Bates appears to be backpedalling a little in the ee article which I take to be the newer one. I dont know why pointing this out is disingeguous

        You tend to excel at this sort of forensic investigation which is why I asked for your opinion.

        Tonyb

      • tony –

        =={ There appear to be inconsistencies between the use of the thumbs on the scale quotes as they appear to have a different context in each of the articles {==

        How did you interpret the following when you read John’s post?:

        –snip–
        Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets— in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.

        […]

        So, in every aspect of the preparation and release of the datasets leading into K15, we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation.
        –snip–

      • Bates isn’t backpedaling but he is putting right a common misperception about what he wrote. Rose and Smith took the misperception and ran with it, so the horse is out of the stable, but at least Bates is trying to repair the damage because it was far from what he intended to happen. That’s what you get for posting on blogs or talking to the wrong journalists.

      • That is correct Willard. And it can cover a wide range of actions. Could Dr. Bates have used a better term? Probably, as the thumb on the scale one could include unethical as well as ethical actions.

        All of this does not make the question of whether there were motivations behind this paper other than scientific ones disappear.

    • Judith ==> National Review has covered the story as well.

    • Fascinating:

      ==> He specified that he did not believe that they manipulated the data upon which the research relied in any way. ==>

      So, putting “thumbs on the scale” but not manipulating data in any way. Very interesting logic, that.

      This is also good:

      =={ “I’m getting much more wary of scientists growing into too much advocacy. }==

      So to avoid any “advocacy,” he makes his argument in perhaps the most political way possible, by publishing it on a highly politicized blog, and working with a highly politicized “investigative journalist,” when it is easily predictable that the sorts of politicization that he disavows would ensue. Certainly, it would be impossible to avoid his argument being politicized – such is the nature of the climate wars – but there are, no doubt, ways that he might have proceeded that would have helped to diminish the magnitude to which his actions aggravate that existing polarization.

      And this is interesting:

      –snip–
      He said he has not communicated with anyone there before and was not a whistleblower for the committee previously but that he expected to be invited to Washington to testify at a future hearing.
      –snip–

      So Judith has been talking about insider whistleblowers for quite a while, and so has Lamar, but John hadn’t previously communicated with anyone on the committee previously.

      Which leads to some interesting questions. If not John, then what “whistleblowers” where Judith and Lamar referring to in their previous statements. If they were referring to John, then who might it be that was acting as middleperson?

      • And of course, this is good also.

        =={ He said he would accept such an invitation, but cautioned scientists against advocating policy. }==

        So he’s going to testify for the committee which is focused on writing policy in order to avoid advocating policy.

        Looks like his definition of advocacy is similar to Judith’s in that it applies only to those who disagree with him about policy.

      • Joshu#

        I am obviously missing something.

        I do not think that sceptics have been tossed a juicy bone or that the story is any sort of revelation on a par with climategate

        It seems a bit of a non story. Retired scientist says another retired scientist didn’t always follow protocols but insists there is no fraud Involved.

        Some of you are fixated on the thumbs on the scale comment.

        In the ce article it appears to have more impact than in the latter ee article where it is little more than a passing reference..

        So, is the thumbs on the scales comment significant or not and should I be getting more excited about the implication for sceptics than I currently am?

        Tonyb

    • I just arrived to pass on that news, amongst other things. For Tony B

      The federal climate scientist hailed by conservatives as a whistleblower for allegedly revealing manipulated global warming data said yesterday he was actually calling out a former colleague for not properly following agency standards for research.

      In an interview with E&E News yesterday, former National Oceanic and Atmospheric Administration principal scientist John Bates had a significantly more nuanced take on the controversy that has swirled since a top House Republican hailed his blog post as proof that the agency “played fast and loose” with temperature data to disprove the theory of a global warming “pause.”

      Bates accused former colleagues of rushing their research to publication, in defiance of agency protocol. He specified that he did not believe that they manipulated the data upon which the research relied in any way.

      “The issue here is not an issue of tampering with data, but rather really of timing of a release of a paper that had not properly disclosed everything it was,” he said.

      • Jim hunt

        I thought that is what I said. Willard has taken me to task for it. As far as I can see bates is questioning protocol rather than suggesting fraud.

        Is the phrase thumbs on the scales significant or not?

        Tonyb

      • Tonyb
        The thumb on the scale issue is using very inaccurate data to adjust accurate data. The issue goes away if ARGO results are used as is an estimated errors bars are used for ship intakes. Ships don’t measure repetively at one time, use mostly whole degree thermometers, take samples from various depths, take samples through varied configuration of piping and heat transfer impacts. In general use the ARGO data and the ship intake data to indicate gross trends, not detailed accurate adjustments.
        Scott

      • tony –

        =={ As far as I can see bates is questioning protocol rather than suggesting fraud. }==

        Perhaps you see it differently, but I would say that most people would consider accusations of “throwing out data” and “constantly” putting “thumbs on the scale” in “every aspect” of preparation and release of datasets in an effort to “discredit” the notion of an hiatus, as being scientific fraud.

      • tony –

        =={ Is the phrase thumbs on the scales significant or not? {==

        I would say that Bates’ repeated use of the phrase suggests he considered it significant.

      • Scott

        As far as I am aware the article concerns the very narrow question of how to correctly convert temperature data from ships to buoys so the two can be compared.

        That is quote a separate matter to the quality of tie data in the first place. You may remember my article on this subject from some hears ago

        https://judithcurry.com/2011/06/27/unknown-and-uncertain-sea-surface-temperatures/

        I think the idea of a valid global oceanic temperature data set to 1850 or so is complete nonsense. Methodology was very poor, readings random , data very limited.

        to try to make this sows ear into a silk purse is futile. Far better to set a realistic start point of around 1960 and base your policy on that. Although we all know that is far too short a time to get any proper ideas f what oceanic temperatures are doing

        Tonyb

      • Tonyb,
        Thanks for the response.

        I remember the article. My point remains, adjusting high quality more intensive data to low quality trend only data is nonsensical. One can say, here is the general trend data we have prior to 1978 and this is the data starting after ARGO and getting more intensive and higher quality as more units are deployed. Adjusting the good data to the estimated grossly inadequate poor data can only be misleading.

        They should remain separate trends. A third trend one could make would be the canvas buckets dropped over the side of sailing ships, hauled up by hand and measured by sticking a glass mercury thermometer calibrated to 2* for a measurement.

        Scott

      • Tonyb,
        Also nice to reread the 2011 article about measures of SST prior to the modern era.
        Scott

      • Mornin’ Tony (UTC),

        Sorry, but I missed this earlier in amongst all the excitement of discovering that GWC is the 21st greatest GW blog on the planet!

        Yes. Of course that phrase (and its synonyms) is important. For legal as well as scientific and political reasons!

        Has John ever uttered words to that effect. Has Judith? Has David Rose? Have any of the Alt-Right meejah? Have any of the “retweeters” and “rebloggers” and other assorted parasites? For more on that thorny topic see:

        http://GreatWhiteCon.info/2017/02/climategate-2-falls-at-the-first-hurdle/#comment-217767

        et seq.

    • Steven Mosher

      This a Better on the Bates Affair

      “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

      Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

      However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

      “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  71. Pingback: La passe d’armes continue sur le « pausebuster  | «Mythes, Mancies & Mathématiques

  72. Whoops – That ended up in the wrong place. Moving on, nobody seems to have responded to last night’s teaser, so without further ado a pertinent news release:

    Speaking from their Ivory Towers near the North Pole, Great White Con spokesperson Snow White announced by the light of the silvery moon:

    We are extremely proud to have been selected as Feedspot’s 21st best Global Warming blog on the web. Whilst it’s galling to be below WUWT we’re well ahead of the GWPF and Climate Etc. is nowhere to be seen.

    By way of celebration we have some Shock News to impart!

    http://GreatWhiteCon.info/2017/02/beta-testing-snow-whites-alternate-fact-detector/

    We flipped the switch on the first beta test version of Snow White’s Alternative Facts Wetware™ (AFW™ for short) AF detection subsystem early on Saturday morning (UTC). We were astonished when the needle literally flew past the end stops later that morning. Initially we suspected a bug must have sneaked in via one of Snow’s unprotected ear canals. However when she rather reluctantly ran her exhaustive diagnostic routines they revealed that her mission was in actual fact absolutely nominal.

    What happened next therefore came as no surprise whatsoever

  73. –snip–
    Climate Home, a nonprofit site based in London that offers news and analysis, also weighed in on one of the central contentions of Mr. Rose’s article, that the publication of the NOAA paper had “duped” policy makers into adopting the Paris accord. The site contacted representatives to the talks from 10 countries; none said that the paper had any influence.
    –snip–

    https://www.nytimes.com/2017/02/07/science/was-data-manipulated-in-a-widely-cited-2015-climate-study.html?

  74. Judith Curry, I have a post in moderation because of a number of links I made to temperatures series I used in my CEEMD analysis posted here and to papers making similar analyses..

  75. Pingback: CFACT -

  76. Pingback: As the planet warms, doubters launch a new attack on a famous climate change study – Enjeux énergies et environnement

  77. Natural, large-scale climate patterns like the PDO and El Niño-La Niña are superimposed on global warming caused by increasing concentrations of greenhouse gases and landscape changes like deforestation. According to Josh Willis, JPL oceanographer and climate scientist, “These natural climate phenomena can sometimes hide global warming caused by human activities. Or they can have the opposite effect of accentuating it.” http://earthobservatory.nasa.gov/IOTD/view.php?id=8703

    This has been investigated for decades – with not a lot of progress it must be said.

    https://watertechbyrie.files.wordpress.com/2016/02/sst_anomaly_amsre_2008105.jpg

    There is something that causes simultaneous shifts in the volume of upwelling in the north-east (PDO) and central Pacific (ENSO) in 20 to 30-year regimes. Changes in sea surface temperatures Sea surface temperature change winds, currents and clouds – as well as global hydrology.

    What evidence there is – ERBE, ISCCP, Project Earthshine – shows declining reflectance in the last decades of the 20th century and an in Earth albedo at the turn of the century.
    Cooler sea surface temps cool the atmosphere with more cloud at lower levels – leading to lower IR emissions and higher SW. And vice versa. A changing Earth energy budget in response to more or less Pacific upwelling – and it changes a lot.

    https://watertechbyrie.files.wordpress.com/2014/06/moys-20023.png

    Christopher Moy and colleagues examined a sediment core from Laguna Pallcacocha in southern Ecuadora. More rainfall and runoff In El Niño conditions wash more red sediment into the lake. So we know it was pretty rainy in South America a 1000 years ago. Some 5,000 years ago there was a change from dominant La Niña anomalies to dominant El Niño – that dried the Sahel. Just 3,500 years ago there were a long series of El Niño with red intensity greater than 200 and civilisations fell. For comparison – red intensity in the ‘monster’ 1997/1998 El Niño event was 99.

    Solar variability – including UV – change polar pressure fields. These drive storm tracks more or less into lower latitudes. They spin up the Pacific gyres driving more or less cold polar water along the Peruvian and Californian coasts and into the areas of upwelling.

    https://watertechbyrie.com/2017/01/12/an-earnest-discovery-of-climate-causality/

    So yes Virginia there was a climate shift after 1998 – the next is due within a decade – we really don’t know where it is going to.

    So basically you have missed whole slabs of critical science to focus on a rate of increase in greenhouse forcing of 0.032W/m2 – and that everything is evidence that it and it alone is responsible for climate change.

    But we still love you and really, really want to talk about alternatives to dismantling western civilisation.

    https://watertechbyrie.com/2017/02/06/the-climate-problem-and-the-solution/

    • … an (increase) Earth albedo at the turn of the century…

      • Chief

        I find reading the similar things you have presented before, now after a time interval not only much easier to grasp, but much more understandable. Some of it is language of course, becoming familiar with word meaning and the concepts these words represent. I find your return to this site brings much insight. Maybe my prior reading was disjointed, laboring as I do on specifics. However, I find your present style very informative, almost as if you have thought through some issues and now articulate them in a way, which to me are much more understandable.

        Your sabbatical has done wonders for my understanding!

      • G’day mate. I’ve done a lot of writing in the past two years – some would say too much.

      • “Your sabbatical has done wonders for my understanding!”

        It helped me too :)

      • I assumed it was trivial snark from one of the usual suspects. Usually a fairly safe assumption. If I have misunderstood – I apologise.

    • I don’t really know you at all Nick – but I’m inclined to doubt that.

  78. @tony banton

    I put it on the line the ‘problem’ and the solution – what’s yours. You may copy and paste from urban doofus hipster climate blogs if you like – although this will be fairly obvious. Here’s a start.

    Intensifying many human activities — particularly farming, energy extraction, forestry, and settlement — so that they use less land and interfere less with the natural world is the key to decoupling human development from environmental impacts. These socioeconomic and technological processes are central to economic modernization and environmental protection. Together they allow people to mitigate climate change, to spare nature, and to alleviate global poverty. Ecomodernist Manifesto

    The alternate doofus vision involves narratives of moribund western economies governed by corrupt corporations collapsing under the weight of the internal contradictions – leading to less growth, less material consumption, less CO2 emissions, less habitat destruction and a last late chance to stay within the safe limits of global ecosystems. Their duty is to bring this about by any means. And this is just in the ‘scholarly’ journals.

    Perhaps a solutions page Ms Curry? You may use this and I may be emotionally resilient enough again to withstand a barrage of abuse and denigration. I could call in some new names for a broader perspective.

    https://watertechbyrie.com/2017/02/06/the-climate-problem-and-the-solution/

    • “I put it on the line the ‘problem’ and the solution – what’s yours. You may copy and paste from urban doofus hipster climate blogs if you like – although this will be fairly obvious. Here’s a start.”

      Robert:
      Like I said – glad to have filled in some of your missing knowledge.
      And I don’t consider https://www.ncdc.noaa.gov to be an “urban hipster climate blog”, and that you do says everything needed to know of you. An obvious CT’er.

      You don’t consider WUWT, a “doofus hipster climate blog” then?
      Strange?

      Tell you what.
      Try going to the above site and downloading the RATPAC A data and plot it against RSS and UAH (latest versions).
      Then come back here and proclaim to the world that those “doofus” graphs are wrong.
      And then we will have the complete message.
      Radiosondes data (as well as being a mystery to you) does/does not correlate with the two major sat trop temp data series.
      As for your other ranting.
      Just more politics of the Trump era.
      Try some science and don’t naysay the obvious – In this case UAH (was RSS before V4.0) is nowhere near the “Gold standard” (JC) of global “surface” temp records.

      • Oh for God sake give it a rest.

      • You are really just flailing around and substituting unsubstantiated and random for any hint of rational discussion.

        You link to a top page of a NOAA site. By all means – show where anything there has any relevance. And yes I know they keep RATPAC data – from 85 stations not hundreds.

        Nor do the calibrate AMSU to radiosonde data – as I said before – and provided this link – http://www.remss.com/measurements/upper-air-temperature/validation It seems to be down at the moment but was working earlier. I take it you haven’t looked at it?

        You strike me as someone who has little understanding of the broad range of relevant primary sources in the complex field that is climate science – and expect to overawe by cutting and pasting memes from the climate wars.

        I’m not a part of the climate war – I am an equal opportunity disdainer.

        So I take it that’s a no on discussing some actual policy options?

        Now I’ll give it a rest.

      • … random pejoratives that is… You even link me with Trump – the ultimate urban doofus hipster insult…

      • I read a Mosher comment above, something to the effect that “sat data keeps getting adjusted”. I think he meant this comment to be a clear cut reason to give little credence to sat data. It’s amusing. I’m certain Mosher understands the challenges of drift and the technical software challenges associated with keeping up improvements. Or maybe Mosher doesn’t like that sat data reads cool compared to the field. In any case, a quick look at surface, balloon, and buoy data reveals far greater challenges. Shall we list them?

    • Geoff Sherrington

      Ehak,
      It is possible because data completely unfit for purpose is being used in a web of very poor climate ‘science’ by people who display little understanding of sampling theory.
      They wrongly assume that an academic/mathematic analysis can correct wrong results. In real, practical life one does meet data sets that cannot be resurrected.
      Ship intake temperatures are clearly such a case. Strip out the emotion, discard the wish to have some data to support an invented story and be objective about the ship data.
      It is not good enough for its present use, is it? Not marginal, but clearly so full of irreversible problems that it should never have been floated from the start.
      People need to get some guts and go public with this magnificent example of the Emperor’s clothing.

    • Geoff Sherrington

      RIE,
      Seasoned participants in this whole environment protection topic are well and truly tired of aged social justice warriors stamping feet, still trying to make a case that the preservation of the environment is more important than established economic activities like primary production, actions that actually feed people and stop death.
      We look forward to the day you realise you have failed to get the status you crave. Don’t look for evidence in the words of trendy media writers, go out to the real world of starving people in poverty which is the home of the huge majority of people.
      Ask them if they give a fig about the nobility of action to place the environment at the top of the urgent list. They do not.
      So write or chatter away, but remember you are inside a small, synthetic kernel. Not the real world.
      Geoff

  79. Steven Mosher

    Judith

    You and Bates and Rose have some Explaining to do and apologies to make

    “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

    Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

    However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

    “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

    Come clean.

    • It’s good to see Mosher hasn’t become any more honest with time.

      Why should Dr.s Curry and Bates apologize for making charges they never made?

      “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus”

      has nothing to do with

      “data tampering”, “data changing”, “maliciousness” or “trumped up data”.

      I understood what Bates referred to regarding “documentation, scientific choices and release of datasets” because he explained it rather clearly.

      For example, choosing to use data that is not fit for the purpose for which you use it is not “data tampering” or “data changing”, it is using data not fit for its intended purpose. Being so eager to disappear the pause in a timely manner that you use data unfit for the purpose you intend it is not maliciousness, it is allowing ego to prevail over proper scientific methodology.

      Using data that you know is unfit for the purpose you use, because it gives you the result you want, it is not tampering with data, but it surely can fairly be called putting your thumb on the scale.

      So now we can await the apology from Mosher to Dr.s Curry and Bates for his falsely accusing them of saying Karl tampered with data, changed data or was malicious, when Karl himself said the opposite.

      I won’t hold my breath.

      • Steven Mosher

        Bates makes a charge of Thumb on the scale.
        Judith Promoted this claim.
        She promoted this claim knowing Full well that Zeke and Rohde had tested K15 against independent sources.
        She promoted this claim knowing full well that skeptics would twist
        this into a claim of fraud.
        She and Bates had an Opportunity to argue clearly and forcefully

        A) that there was no evidence of fraud. The science is sound
        B) that there was evidence that a more rigorous process could have been followed.

        Instead they made the argument that Bates process wasnt followed and Karls thumb was on the scales.. This presentation leads Directly to charges that they know to be false.

        Now they try to back peddle and claim innocence because they never uttered the actual words.

        They sold a firearm to mental defectives and now pretend they dont bear an ounce of responsibility.

        Their Thumb on the PR scale is pretty heavy

      • It’s a political knife fight, Mr. Mosher. CAGWers started it.

        Get over it. All the verbal dust you throw up is simply an attempt to put lipstick on that AGW pig.

      • > choosing to use data that is not fit for the purpose for which you use it is not “data tampering” or “data changing”,

        It does sound like “Trumping data” however.

        Imagine Karl’s thumb. It cannot be smaller than the Donald’s.

        And we all know what teh Donald can grab with the undersized hand opposite to that tiny thumb.

        Imagine Karl’s thumb with the data from the paper that sunk Judy’s Paws and that took billions of dollars away from the workers and the poor.

        By God, let’s never forget the poor.

        Americans first, of course.

        Then all the others.

        All we need is to get Karl’s thumb out of the balance.

        Only Karl’s emails will save us now.

        The balance of INTEGRITY ™ is at stake.

        Thank you.

      • Willard, thank you for affirming a belief I have held about you for some time: Your political views infect everything you touch.

      • And what are my political views, Charlie?

      • “She promoted this claim knowing full well that skeptics would twist this into a claim of fraud.”

        “This presentation leads Directly to charges that they know to be false.”

        “Now they try to back peddle and claim innocence because they never uttered the actual words.”

        Yer a funny guy. And not in a comical way.

        You accuse them of charging Karl with data tampering, data changing and maliciousness. Hilariously, you use Karl’s own words saying that didn’t happen, to imply that the true charges he did make were the same as the charges he specifically says he never made.

        You made a false accusation. When caught, you now claim that the truth that Curry and Bates wrote could be misinterpreted by others to make false charges, and therefore they are “guilty” of saying what you now admit they never said.

        You should stick to screwing up statistics. You suck even worse at logic.

      • > You accuse them of charging Karl with data tampering, data changing and maliciousness.

        Your paraphrases won’t cut it anymore, GaryM.

        Citation needed.

      • I guess GaryM has not read the Rose article that purports to carry Bates’s views. Lamar Smith did and was misled by it. Rose has made-up stuff to answer for, but he has always been fast and loose with the facts, and Bates should have been warned of that before agreeing to an interview.

      • Try Senior‘s drive-by, JCH:

        John- Thank you for doing this. Your experience with inappropriate behavior by Tom Karl, Tom Peterson and Peter Thorne at NCDC is consistent with my experiences with the CCSP 1.1 Report. If you have not read these, you might find them informative

    • Mosher needs to come clean.

      Mosher is putting words in Bates’ mouth that were never said. What he said in the post was said in the interview.

      Mosher is suggesting Bates said he didn’t think it was warming. He said no such thing.

      Mosher implies that Bates implies there was data tampering. Bates said no such thing.

      Mosher owes the world an apology for creating deception and putting words in Bates’ mouth that he never said.

      • Steven Mosher

        Bates says Karl’s Thumb was on the scales.
        Curry Promoted this claim.

        If having your Thumb On the scales is NOT a metaphor for cheating the results What is it a metaphor for?

        https://en.wiktionary.org/wiki/thumb_on_the_scale

        In short.

        Bates used the metaphor Thumb On the scale. That is a Metaphor for CHEATING YOUR CUSTOMER.

        Curry promoted this metaphor with her good name.

        Now we find out that it is Not about Cheating your customer or misleading folks in Paris, we find out that its about Bates paperwork.

      • “Mosher implies that Bates implies there was data tampering. Bates said no such thing.”

        The headline of the Rose article, which Judith and John Bates seem to have had a considerable hand in, had the banner headline:

        “Exposed: How world leaders were duped into investing billions over manipulated global warming data”

        and David Rose is an award winning investigative journalist

      • Rose’s article and Bates posting here are getting a lot of references, and Rose is not coming out well in it. Smith even less. Bates is setting the record straight in E&E News (but you need to subscribe).
        http://asktran.com/home/2017/02/07/rep-lamar-smith-tried-to-make-a-big-deal-out-of-a-fake-climate-scandal/

      • Actually the E&E News article is available.
        http://www.eenews.net/stories/1060049630

      • I suspect Bates was horrified by the Rose article and further dumbstruck by Smith’s action on it. What a mess he got his name into. Now he’s doing damage control. Judith instigated all this.

      • “If having your Thumb On the scales is NOT a metaphor for cheating the results What is it a metaphor for?”

        It was a metaphor for using data not fit for making massive policy decisions, to argue for those policy decisions.

        How do I know that was what the metaphor was to show? Because bates said that was what it was to show.

        ““constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets”

        But Mosher knows this. He quoted the damn language. But he got caught making false claims again, and so we get billowing clouds of obscurantist BS.

        Apology? Mosher?

        Ahahahahahahahahahahaha.

      • “Exposed: How world leaders were duped into investing billions over manipulated global warming data”

        You don’t have to tamper with data to dupe progressive politicians.

        (They don’t need to be duped anyway. Karl was really just giving them ammunition for CYA purposes so they could do what they wanted to do anyway.)

        You can dupe people by using crap data and pretending that it isn’t crap data,

      • > It was a metaphor for using data not fit for making massive policy decisions […]

        Perhaps you missed what followed the metaphor, GaryM:

        Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets—in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.

        https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/

        While waiting for that evidence that kept mounting, pray tell more about progressives.

      • GaryM –

        =={ It was a metaphor for using data not fit for making massive policy decisions, to argue for those policy decisions. }==

        Really? So how might we check on whether your interpretation is biased? Perhaps we could check by looking to see how other people interpret the metaphor?

        Federal scientist cooked climate change books ahead of Obama presentation, whistle blower charges

        http://www.foxnews.com/science/2017/02/07/federal-scientist-cooked-climate-change-books-ahead-obama-presentation-whistle-blower-charges.html

        Don’t forget to watch the video, where you hear the explanation that the data was “manipulated” by scientists. Indeed, that NOAA intentionally manipulated data to hide a 12-year pause in global warming.

      • Willard,

        I didn’t miss what followed, I actually read it. More importantly,
        I understood it and felt no need to misrepresent it.

        I also read the two sentences that preceded it. Did you? (If not, see below).

        Perhaps you can show us where the words “tampering”, “changing” or “malicious” appear?

      • “where the words “tampering”, “changing” or “malicious” appear”
        There’s plenty of that talk in Rose’s article. Judy was the one who got Rose and Bates together. I mentioned the banner headline above:

        “Exposed: How world leaders were duped into investing billions over manipulated global warming data”

        Reading on

        Dr John Bates’s disclosures about the manipulation of data behind the ‘Pausebuster’ paper is the biggest scientific scandal since ‘Climategate’ in 2009 when, as this paper reported, thousands of leaked emails revealed scientists were trying to block access to data, and using a ‘trick’ to conceal embarrassing flaws in their claims about global warming.

        Both scandals suggest a lack of transparency and, according to Dr Bates, a failure to observe proper ethical standards.

        And of course, Lamar Smith

        “‘for courageously stepping forward to tell the truth about NOAA’s senior officials playing fast and loose with the data in order to meet a politically predetermined conclusion’”

      • GaryM,

        There were two uses of ‘thumb on the scale’ in Bates’ essay. Here’s the other one:

        ‘we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation.’

        If Bates and Curry want to disassociate themselves from any accusation of malicious wrongdoing and claim that these statements were misinterpreted, all they need to do is publicly condemn Rose’s article and apologise for the misleading reportage which came out of their involvement with it.

      • > Perhaps you can show us where the words “tampering”, “changing” or “malicious” appear?

        These words appear in Bates’ quote, not in Moshpit’s accusation.

        Could you provide a quote that would illustrate what you may consider Moshpit’s accusation?

        Many thanks!

      • paulskio,
        And that second use says nothing about “tampering”, “changing” or “maliciousness”.

        Willard,
        Mosher used to second quote to show that the first quote was so wrong and unfair that they both owed Karl an apology. But you knew that already. (You might notice – you knew that already too – that Mosher never denied that was his accusation, he just has twisted himself in knots trying to justify it.)

        Nick Stokes,
        Dr. Curry is responsible for what she says. I don’t blame you for Mosher’s lies. Nor does your quote from the congressman equate with “tampering”, “changing” or “maliciousness”.

        Cherry picking your statistical assumptions based on whether they increase or decrease GAT, is not tampering or changing data. Nor does it require maliciousness. But it can fairly be called putting your thumb on the scale or even playing fast and loose with the data.

        Here endeth the lesson in reading comprehension for the ideologically blinded.

      • > Mosher never denied that was his accusation

        Nice burden of proof reversal, Counsellor. The Moshpit has yet to show any concern for your minimization attempts. Why do you think deflection will work?

        It should be easy to quote specific accusations the Moshpit made. Yet you refuse to do any kind of work. You just stand there and harangue.

        It’s boring. It makes you look silly. And in this case, it’s built on #AlternativeFacts.

        Please desist.

      • “Don’t forget to watch the video, where you hear the explanation that the data was “manipulated” by scientists. Indeed, that NOAA intentionally manipulated data to hide a 12-year pause in global warming.”

        I don’t see how you can read what Bates said and not conclude that data was purposefully manipulated in an attempt to erase the pause.

        This is what Bates said, not Dr. Curry or Rose.

        DO we need lesson in plain English here. Bates said it and later wished he hadn’t.

    • Hear, hear Mr. Mosher!

      I had a nice brief chat with IPSO yesterday and then a nice long chat with Geordie Greig’s PA this morning. For some strange reason I’ve not heard back yet.

    • Mosher, maybe JC should wait until the emails are released to make her apology. And if the correspondences at NOAA validate JC’s decision to let this mess play out on her blog…will you come clean and toast to her credibility?
      It’s interesting, you seem wrapped up in the he said she said, when optics make the critical case. And it looks bad. This story will play out similarly at NASA before long. Why? Because political appointments populate ranking scientific posts.

      • Steven Mosher

        No the apology is due now.
        Bates for making vague accusations of manipulation.
        Curry for not asking him the pointed questions that folks like the AP. Did.

        Medium sized infraction. Apology is preferable to ugly lawsuit.

      • Steven, We dont know exactly how all of this is unfolding. What we do know is that adjusting buoy data raises eyebrows, yours included. It’s easily enough to claim opperational bias. For gods sake, karl’s paper led to a speedy adjustment in ersst. If anything there exists a case for collusion up the chain. This has been suspected in force since K15 came out. Im curious as to why certain people think this is off base? Malfeasance is not unheard of in Government. And should Karl be vindicated im sure we’ll all feel stupid and appologies will follow

      • “Apology is preferable to ugly lawsuit.”

        Mosher,

        It’s kinda creepy that you are making legal threats.

        Then again, creepy and you are already familiar.

        Andrew

      • Why would Tom Karl and Tom Peterson cheat? It’s absurd on its face.

        Some people around here need to start thinking rationally.

        Hopefully the speaker of the house will tell Lamar Smith this is over and done. That is what is supposed to happen when a committee chair gets drunk with power.

      • JCH, Who knows why. There’s plenty gor us to speculate on.

      • Speaking of creepy…

        Andrew

  80. Steven Mosher

    Zeke and Robert Rohde weighed “Karls thumb” they found what Bates Already Knew.

    It was weightless

    “The hubbub was sparked when retired NOAA data scientist John Bates claimed in a blog post that his boss, then-director of the National Centers for Environmental Information Thomas Karl, “constantly had his ‘thumb on the scale’ — in the documentation, scientific choices and release of datasets — in an effort to discredit the notion of a global warming hiatus” and rushed a study published in the journal Science before international climate negotiations.

    Bates said in an interview Monday with The Associated Press that he was most concerned about the way data was handled, documented and stored, raising issues of transparency and availability. He said Karl didn’t follow the more than 20 crucial data storage and handling steps that Bates created for NOAA. He said it looked like the June 2015 study was pushed out to influence the December 2015 climate treaty negotiations in Paris.

    However Bates, who acknowledges that Earth is warming from man-made carbon dioxide emissions, said in the interview that there was “no data tampering, no data changing, nothing malicious.”

    “It’s really a story of not disclosing what you did,” Bates said in the interview. “It’s not trumped up data in any way shape or form.”

  81. Steven Mosher

    Update of global temperature comparisons since 1979, now including Copernicus reanalysis (based on ERA-Interim): pic.twitter.com/HNFjPhF336— Zeke Hausfather (@hausfath) February 7, 2017

    //platform.twitter.com/widgets.js

  82. Pingback: Politico-Scientific establishment rushes to climate alarm at The Global Warming Challenge

  83. I think Gary M hit it. Politics transparently in climate science for years. Lots of thumbs weighing. Details here just a tempest in a teapot. Maybe just meant to shine light where it’s not welcome. Surface temperature records seem a hopeless convoluted homogenized mess. Individual non corrected longitudinal records best for any real insight there. Satellites the only hope for global temp… even if not surface. Difficult too, but much better coverage and better more quantifiable accuracy and precision. And is global temperature really a meaningful measure? Supporting hubris of ensured doom decades hence with unsuitable data is an underlying issue. We have a highly complex, nonlinear water world climate, that if anything seems to have nothing but tipping points with many countervailing, and robust, checks. The more one digs, higher risk potential by unnecessarily forcing highly regressive cures on the world becomes the ever more convincing scenario than the touted virtual future harm expectations, with what ongoing is looking to likely be a more resilient, and “self determined” climate than promoted. Suppression of dialogue, scientific and political, has driven this to religious fervor for many, with scientific and political elite largely responsible. It has raised the very real possibility that “truths” uncovered by much wonderful research are tainted. Climate science has a serious credibility issue with many, perhaps a majority. Less politicized science and more objective, inclusive dialogue surrounding our future can only be a good thing. Perhaps whistle blowers will prove to be the catalyst?

    • > Perhaps whistle blowers will prove to be the catalyst?

      Of course it will, new Denizen. Doc’s already smelling the popcorn. The tide is turning as we speak:

      John- Thank you for doing this. Your experience with inappropriate behavior by Tom Karl, Tom Peterson and Peter Thorne at NCDC is consistent with my experiences with the CCSP 1.1 Report. If you have not read these, you might find them informative

      https://judithcurry.com/2017/02/04/climate-scientists-versus-climate-data/#comment-836536

      John’s message is loud and clear.

      At least to one esteemed reader.

      There was inappropriate behavior.

      At least by a thumb on a scale.

      It’s time we put less thumbs on scales and we start a Stadium Wave of objective science.

      Let’s Make INTEGRITY ™ Great Again!

      Thumbs up if you agree with me.

      Thank you.

  84. Just for giggles, here is the full paragraph in which the dreaded “thumbs on the scale” comment was made:

    “I questioned another co-author about why they choose to use a 90% confidence threshold for evaluating the statistical significance of surface temperature trends, instead of the standard for significance of 95% — he also expressed reluctance and did not defend the decision. A NOAA NCEI supervisor remarked how it was eye-opening to watch Karl work the co-authors, mostly subtly but sometimes not, pushing choices to emphasize warming. Gradually, in the months after K15 came out, the evidence kept mounting that Tom Karl constantly had his ‘thumb on the scale’—in the documentation, scientific choices, and release of datasets—in an effort to discredit the notion of a global warming hiatus and rush to time the publication of the paper to influence national and international deliberations on climate policy.”

    Notice how when the progressives quote this paragraph, they leave out the first two sentences?

    It’s makes it so much easier to lie about what Bates wrote.

    • “Notice how when the progressives quote this paragraph, they leave out the first two sentences?”

      I cited both of them here as examples of office gossip. And I showed in the previous thread that the 90% was a nonsense. It shows that Bates really doesn’t know much about the issue, and doesn’t seem to have read the paper (nor, if correctly reported, did the co-author). Right there, in Fig 1, K15 explains why:

      http://www.moyhu.org.s3.amazonaws.com/2017/02/k15y.png

      And indeed, in the AR5 in he introduction to Chapter 2 on observations, it states the rule followed in the chapter

      http://www.moyhu.org.s3.amazonaws.com/2017/02/k15z.png

      This is basic. Bates really should know it.

      • My point wasn’t whether Bates was right or wrong, but that Mosher at least has falsely accused bates of making charges of data tampering, data changing and maliciousness. He took the “thumbs on the scale” statement out of context, because in context, Bates’ meaning is clear.

        Bates could be completely wrong, and Mosher still would be lying by claiming that Bates and Dr. Curry made false accusations against Karl.

      • “Bates could be completely wrong, and Mosher still would be lying by claiming that Bates and Dr. Curry made false accusations against Karl.”

        So when we have the Chairman of the House Science Committe thundering about “NOAA’s senior officials playing fast and loose with the data in order to meet a politically predetermined conclusion” there’s no way anyone here could be responsible?

      • > My point wasn’t whether Bates was right or wrong, but that Mosher at least has falsely accused bates of making charges of data tampering, data changing and maliciousness.

        I don’t recall Moshpit ever *using* these words in this thread.

        He was *mentioning* them.

        By QUOTING Bates.

        Over and over again.

        To make sure he backpedaled from his thumbs and his scales.

        More importantly that all Denizens do.

        But the thumbs and the scales and everything is still there.

        You’re peddling #AlternativeFacts, GaryM.

      • Nick

        Did you see the comment by Paul Matthews regarding Steve McIntyre finding very significant instability in the algorithm resulting in 4 degrees C difference at one location?

        That may be the most significant comment of post. I would like your reaction to this troubling development.

      • “your reaction to this troubling development”
        Done, above. Though it isn’t a development about K15, and isn’t particularly new.

      • Nick, imagine a collusion. Karl reveals convenient bias in buoy and fleet data. Corrects both resulting in a pause busting graph. NOAA updates it’s product based on Karl’s findings. All Karl has to do is justify his shoddy work to a few well meaning peers and presto…no pause. But the maneuver was an obvious one, and it caught our attention. It caught Lamar’s attention, and that is all the story can be until internal conversations are studied. It looks bad. Do you not see it? Let the left brain take a Crack at the narrative for a few minutes. Can you do that? The vet least we can do is investigate the hell out of these allegations and see what turns up.

      • Nick

        I understand it is not directly related to K15. But in terms of the general confidence in the data or lack thereof there is a clear nexus. My point was the potential impact from dodgy algorithms could have larger and more long lasting reverberations than the Bates issue.

        It is interesting that if this is not new, McIntyre would express surprise. Surprise usually is not the reaction someone has when they already knew what ever they were surprised about.

      • “Karl reveals convenient bias in buoy and fleet data. Corrects both resulting in a pause busting graph. “
        Karl didn’t reveal it. It was known since at least 2008, and well quantified by Kennedy et al 2011. A correction had to be made; the only question is, why did it take so long. It seems part of the answer is – John Bates.

        Scientists try to get stuff right. If they see something somewhat unexpected, they will look to see if there is a bias or other difficulty in the measurements. They don’t ask whether pause-lovers feelings will be hurt. They get it right.

      • Karl used ERSSTv4 which had the pause-busting buoy correction. Karl was not an author of that dataset, just the first person to mention its pause-busting consequence. Note that Bates said he wanted ERSSTv4 out earlier than Karl in his post because it had already passed all of his own controls. Many have not yet realized these aspects of the story which fundamentally change how you would think of Bates and Karl’s role.

      • Nick Stokes said, February 8, 2017 at 9:12 pm:

        “Karl reveals convenient bias in buoy and fleet data. Corrects both resulting in a pause busting graph.”

        Karl didn’t reveal it. It was known since at least 2008, and well quantified by Kennedy et al 2011. A correction had to be made; the only question is, why did it take so long. It seems part of the answer is – John Bates.

        Scientists try to get stuff right. If they see something somewhat unexpected, they will look to see if there is a bias or other difficulty in the measurements. They don’t ask whether pause-lovers feelings will be hurt. They get it right.

        Problem is, Nick, that in the case of ERSSTv4 they got it WRONG. They took something real (“The Pause” in global temperature rise between 1997/98 and 2013/14) and artificially made it go away. So either their science is lacking, or it’s agenda driven (biased, that is) …

      • “Scientists try to get stuff right. If they see something somewhat unexpected, they will look to see if there is a bias or other difficulty in the measurements. They don’t ask whether pause-lovers feelings will be hurt. They get it right.”

        This might be the most pathetic appeal to labcoattery I have ever read.

        Andrew

    • Rather amazing how people can’t read or can’t understand that terms and phrases might have more than one meaning.

      Also amazing how they think that one doesn’t make decisions every day regarding what methods to use, what data to keep and which can be thrown out, etc. etc. Nothing unethical, no tampering. The choice on confidence level 90% rather than 95% is one. One can argue that it is not good practice, but not that it is tampering. It’s simply a choice. Was it a choice with a certain end result in mind? Possibly. And when the entire process gets laid out and we can see all of the choices that were made, it may be possible to identify patterns, even if each decision on its own was not unethical. If such a pattern exists, then yes, one can reasonable say that a thumb was on the decision scale.

      Every good researcher should know that when choosing what data to use and what to discard, they could possibly be invalidating their conclusions. Or that the outliers they discarded could be far more informative than the data they kept. And that is with no thumb on the scale.

  85. Pingback: Climategate 2 ? - Climategate.nl

  86. Pingback: Response to critiques: Climate scientists versus climate data | privateclientweb

  87. How naiv is it possible to be? Arming David Rose with “thumb on the scale” and believing that the hordes over at WUWT shall read it in context?

    • Don’t forget “throwing out data.”

      • Geoff Sherrington

        You mean “Throwing out data, later to be brought back in and adjusted.” That is how it went. It did not go to the trash bin.

    • So, in other words, the data weren’t thrown out so the statement was false.

      Btw… If you went to the butcher and paid for more meat than what you took home because she put her put his thumbs on the scale, would that mean you were cheated?

    • If all you have is David Rose, then you have nothing. Try focusing on what John Bates said in the post. Complaining about what a journalist wrote is trying to divert attention.

      Besides, don’t see you complain when it is Seth Borenstein, Bob Ward or John Abrahams.

  88. Pingback: Whistleblower Reports Data Tampering At NOAA | PA Pundits - International

  89. There’s an elephant in the room that none of the deniers of climate science ever seem to want to address.

    For years there has been incessant regurgitation of the notion of a pause. This posturing consistently and completely ignores the contributions of the magnitude of short-term variability (or ‘noise’) around the underlying trend, and of the magnitude of the trend rate itself, and how these two parameters dictate the minimum period of time required to statistically discern the actual signal, and why all this makes a mockery of the suggestions of a ‘pause’.

    But this isn’t the elephant. Well, it an elephant, a great big bull elephant, but it’s not the particular one to which I referred in the first sentence of this post.

    That elephant is the nature of the reliability of the temperature record, which causes so much incensed umbrage amongst the science-denying community and which is the primary motivation for the current fulmination against Karl et al. It’s the insistence is that a warming signal cannot be discerned because the data are unreliable.

    Really? Then can someone explain how they know, conversely, that there’s actually a ‘pause’? If the data are unreliable, or corrupted, or mishandled, or whatever other excuse is proffered, how can the deniers use the very same data to claim a ‘pause’? What is the methodology that identifies the ‘pause’ in this apparently substandard data, where that same data cannot be relied upon to describe the existence of human-caused global warming?

    A proper methodology please, just as has been (spuriously) demanded of Karl et al and all the other professional scientists that already actually explain their work. A methodology that accounts for the apparent unreliability of the data, and also explains how the magnitudes of noise and signal, which dictates minimum times required to detect trends, can be differentiated from a ‘pause’. But really, I would like to know how the contemporary global temperature data are good enough to detect a ‘pause’, but not a warming trend.

    Bonus points for anyone who can explain the irony inherent in the apparent ability to detect a ‘pause’ more easily than to be able to detect a trend within a short sample-period of data.

    Note: pointing at cherry-picked El Niño years as a bound for a ‘pause’ period won’t cut the mustard, even if deniers think that it does.

    • Well Bernard if you want data, here’s all the weather stations in NCDC’s global summary of days rolled into all sorts of different groupings, sliced and diced. The only requirement, was the number of samples collected per year. One is >=360 days the other >=365, that is the extent of the station filtering, and by year and area of course.
      https://sourceforge.net/projects/gsod-rpts/files/Reports/Ver%203%20beta/

      Previous version of code is available, I’m still working on the latest version, trying to improve run time and ease of use.

      • I don’t actually “want data” – I have that in spades. What I’m seeking is a climate change deneir who can actually show the appropriate statistical working that discriminates between an actual ‘pause’, and the noise inherent in time series datasets that are impacted by multiple phenomena.

        And I’m just as curious to discover why the quality data produced by professional climate scientists is apparently inadequate to support the discernment of a warming trend, but entirely sufficient to use in the (analytically unevidenced) claims of a ‘pause’ in said warming.

      • I have added a lot of analysis to the raw data, including a uncertainty calculation. The measurements are quite clear, it’s just that they make so much up.

    • The real Bernard, it’s a simple linear regression. Starts now and ends when ithe average deviates. There. All is explained

      • Really?! a “simple linear regression” with cherry picks on top?! You might as well use two thumb tacks and a piece of string as your analysis tools.

        I can only presume that you are a being ironic in the poe sense – or failing that, that you’re as clueless as I alluded to in my previous post. Ask Judtih Curry to explain with simple words why you’re wrong. Of course I’m presupposing that she can – up to and including your response, I’ve never once seen a climate change denier address the substance of the questions I posed. Not a single one seems to be able to explain the difference between the properties of data noise and the detection of actual trend rate changes in time series, nor can they explain why it’s possible to reliably point at a ‘pause’ but not at a warming trend, using the same datasets.

      • Therealbernardj, you cant cherry pick a linear regression. It starts now…now! And it ends when the averages diverge. The end point is constantly changing month to month and so is the starting point.

    • I think you are confused. This elephant is entirely on the climate scientist. The pause or hiatus was an way of spinning lower than expected temperatures. Yes some skeptics have used the wording the climate scientists used in their literature to poke holes in the warming meme, but the wording originated with the climate scientists and is inconsistent with the skeptic argument. A pause or hiatus infers that something was happening that somehow stopped and that will continue some time in the future. The skeptic argument is that changes in temperature due to CO2 are either;

      1. Nonexistent (I am not in this camp).
      2. Not significant enough to be measured.
      3. Significant enough to be measured, but not significant enough to be worried about.
      4. Could be significant enough to be worried about but our best option is not mitigation (other than no regrets policies). (My interpretation of Dr. Curry’s position)
      5. Some combination of the above or some where in-between.
      6. Others?
      None of these require a pause or hiatus.

      The onus is on the climate scientist to show that CO2 causes warming that is significant beyond natural background variation. If you look at the graphs that Dr. Curry provides for temperatures before and after CO2 levels significantly changed, the slopes of temperatures are not significantly different. If the slope has not changed then there is no acceleration as Willard claims below. Further, we know from past data that the temperature today is within the normal range for the planet. Therefore if the slope, acceleration and the value is not significantly different than in the past, where is the evidence that we can measure temperature accurately enough to show that CO2 has made a difference?