by Robert Rohde, Zeke Hausfather, Steve Mosher
Christopher Booker’s recent piece along with a few others have once again raised the issue of adjustments to various temperature series, including those made by Berkeley Earth. And now Booker has double-downed accusing people of fraud and Anthony Watts previously insinuated that adjustments are somehow criminal .
Berkeley Earth developed a methodology for automating the adjustment process in part to answer the suspicions people had about the fairness of human aided adjustments. The particulars of the process will be covered in a separate post. For now we want to understand the magnitude of these adjustments and what they do to the relevant climate metric: the global time series. As we will see the “biggest fraud” of all time and this “criminal action” amounts to nothing.
The global time series is important, for example, if we want to make estimates of climate sensitivity or if we want to determine how much warmer it is today than in the Little Ice Age or if we want to compare today’s temperature with the temperature in the MWP or Holocene or if we want to make arguments about natural variability or anthropogenic warming.
Figure 1. Unadjusted data results are shown in the blue curve. The green curve shows the results if only metadata breakpoints are considered. The red curve depicts all adjustments.
As Figure 1 illustrates the effect of adjustments on the global time series are tiny in the period after 1900 and small in the period before 1900. Our approach has a legacy that goes back to the work of John Christy when he worked as state climatologist: Here he describes his technique.
The idea behind the homogenization technique is to identify points in time in each station’s record at which a change of some sort occurred. These are called segment break points. A single station may have a number of segment break points so that its entire record becomes a set of segments each of which requires some adjustment to make a homogeneous time series. Initially, segment break points were identified in every case when one of the following situations occurred: (i) a station move, (ii) a change in time of observation, and (iii) a clear indication of instrument change.
The results of Berkeley adjustments are shown in the green curve. In addition, we break or slice records where the data itself suggests a break in the record. We refer to this as empirical breaks. This is shown in red. The impact of adjustments on the global record are scientifically inconsequential.
On smaller spatial scales, however, we can see there are certain areas where we could pick stations to show two opposite different conclusions: we could show that adjustments warm the record; and we could show that adjustments cool the record. First, a chart for people interested in accusing the adjustment algorithm of warming the planet:
Figure 2. Adjustments for the contiguous US
And next a chart for those who want to accuse the algorithm of cooling the planet
Figure 3. Africa adjustments
The differences between the various approaches—unadjusted, metadata adjusted, and full adjustment is shown below in figure 4 along with data for selected regions.
Figure 4. The top panel depicts the difference between all adjustments and no adjustments. The black trace shows the difference for all land. Blue depicts USA; red Africa; and green Europe. The lower panel depicts the difference between all adjustments and metadata only adjustments.
As the black trace in the upper panel shows the impact of all adjustments (Adjusted-NonAdjusted) is effectively zero back to 1900. And prior to that the adjustments cool the record slightly. However, we can also see that the adjustments have different effects depending on the continent you choose. Africa, which has 20% of the land area of the globe has adjustments which cool it from 1960 today. While the US, (around 5% of all land) has adjustments which warm its record.
Spatial maps show the same story. Certain regions are cooled. Other regions are warmed. On balance the effect of adjustments is inconsequential.
Figure 5. The effects of adjustments on 2014 temperatures
Figure 6. The effects of adjustments on the last 14 years
Figure7. The effect on trends since 1900
Figure 8. The effect on trends since 1960
Since the algorithm works to correct both positive and negative distortions it is possible to hunt through station data and find examples of adjustments that warm. It’s also possible to find stations that are cooled.
One other feature of the approach that requires some comment is the tendency of the approach to produce a smoother field than gridded approaches. In a gridded approach such as Hadley CRU, the world in carved up into discrete grids. Stations within the grid are then averaged. This produces artifacts along gridlines. In contrast, the Berkeley approach has a smoother field.
If we knew the true field perfectly, we could decide whether or not our field was too smooth or not. But without that reference we can only note that it lacks the edges of gridded approaches and tends to have a smoother field.
One approach to evaluating the fidelity of the local detail is to compare the field to other products using other methods and source data. In this poster , http://static.berkeleyearth.org/posters/agu-2013-poster-1.pdf , we compared our field with fields from reanalysis, satellites , and other data producers. What we see is essentially the same story. Over the total field the various methods all produce similar answers. Locally, however, the answers vary. All temperature fields are estimates, spatial statistical estimates. They all aim at producing a useful global results. They succeed. Deciding which result is also useful at the local level is an area of active research for us.
One final way to compare the results of various data producers is by comparing the spatial variability against that produced by global climate models. While not dispositive this comparison does indicate which temperature products are consistent with the variability found in simulations and those which are less consistent.
Figure 9. Spatial variability. GCM results are depicted in blue. Black lines on the GCM results indicate the variations across model runs of the same model
The vertical axis provides a measure of how much variability in temperature trends is observed across the whole field. The homogenized Berkeley, NASA GISS and NOAA, all broadly agree with historical global climate model runs on this metric. The horizontal axis provides a measure of the local variability in trend (i.e. the average change in trend when travelling a distance of 750 km). On this metric, Berkeley, NASA GISS and NOAA are all consistent with GCMs but on the low side of the distribution.
In general, noise and inhomogeneities in temperature data will make a temperature field rougher while homogenization practices and spatial averaging will make it smoother. Since the true temperature distribution is unknown, determining the right amount of homogenization to best capture the local details is challenging, and an active area of research. However, as noted above, it makes very little difference to the global averages.
In summary, it is possible to look through 40,000 stations and select those that the algorithm has warmed; and, it’s possible to ignore those that the algorithm has cooled. As the spatial maps show it is also possible to select entire continents where the algorithm has warmed the record; and, it’s possible to focus on other continents were the opposite is the case. Globally however, the effect of adjustments is minor. It’s minor because on average the biases that require adjustments mostly cancel each other out.
JC note: As with all guest posts, please keep your comments civil and relevant
Thanks, guys. Now we can move on to the strongly positive water vapor feedback assertion. Where is the evidence?
I dont vape
You wouldn’t.
Kidding aside, it’s an impressive effort. You convinced me some time ago that BEST is about as good as we are going to get with what we have to work with in the surface data realm.
I still have doubts about the accounting for UHI. There were 2 billion people in 1935, since then 5 billion have been added in ever higher concentrations around cities towns villages etc. And that’s where the thermometers are located. Let’s envision those 2 billion people living primitively in the open with their campfires and cookfires. Well now we have 7 billion living in the same space with bigger fires and firetrucks too. I don’t believe that UHI is negligible. But in the grand scheme of things, maybe it’s still negligible. The water vapor is the thing.
You and the other guys who work on this for free deserve commendation. Not condemnation.
Don,
Regarding UHI, we’ve done a fair bit of work looking for it.
See my paper on the U.S., for example:
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/hausfather-etal2013.pdf
Or our AGU poster from a few years back:
http://wattsupwiththat.com/2011/12/05/the-impact-of-urbanization-on-land-temperature-trends/
There is still more to do, but asofyet we haven’t found a particularly large effect globally.
Thanks for your efforts, Zeke. Keep looking. I think it’s there, somewhere:) But I have been wrong, back in 1987.
Yeah. Keep looking, Zeke. Don thinks it’s there. Sure, you’ve done a fair amount of looking and haven’t found it. But Don thinks it’s there. He hasn’t actually looked. But he thinks it’s there so you should keep looking. If you don’t find it in this universe, try a couple of others just in case.
You are overwrought, overeager and overselling it, joshie. Study Henny Youngman. Get back to us in about 4 years.
Don.
Thanks.
I would say the most useful thing that skeptics can do in terms of focusing their collective brain power is to look at UHI.
Since 2008 zeke and I have spent tons of hours looking at adjustments and UHI.
There is no story in adjustments. I mean a story that makes science headlines and not blog headaches. Its the wrong tree to bark up.
There might be a story in UHI. but it will take tons of work.
and that means putting on a different hat. it means putting your own ideas to the test.
Don Monford wrote:
“You convinced me some time ago that BEST is about as good as we are going to get with what we have to work with in the surface data realm.”
+1
And I used to be very suspicious indeed.
I think that Booker and Delingpole suddenly jumping all over this will eventually reflect badly on them. Which is a shame, as they have previously shone a spotlight on some terrible scientific malpractice by alarmists. I think they hope the adjustments are the silver bullet that will finally kill CAGW alarmism. They aren’t.
Jonathan
I am not necessarily disagreeing with you or Don. I think Mosh and Zeke have integrity, but they sure don’t translate that into intelligible or useable information.
If Booker or Delingpole are reading this I can’t see anything that has been written that they could take away and say ‘we were wrong.’
The global record suffers from wildly different levels of accuracy-the record from Albania is not equal to the record for Australia for example.
The methodology to collect the temperature data in the past is often highly variable. The numbers that are used are often as ‘anecdotal’ as my written historic records are claimed to be.
The levels of certainty and accuracy proclaimed by all the data sets are by no means warranted. Global temperature to 1850? How? Why?
The Climate undoubtedly varies. It was not virtually static for 1000 years as shown in the paleo proxy reconstructions and stated by the Met Office on their web site until around 2012. Looking at all the data it appears we are mostly a little warmer than we were during the LIA. Why is that surprising?
Less certainty and more humility over the basic data would not go amiss and a clearer rebuttal of the claims made in the various articles cited here would help this particular sceptic to maintain his belief that there is no hoax, conspiracy or fraud. What I have come to recognise over the last ten years though is that there is an overbearing and unwarranted certainty, when frank admissions that ‘we don’t know’ would be more realistic.
tonyb.
Tony,
In general I agree with you. The real problem is the quality and coverage of the raw data in the first place. The adjustments are, to paraphrase Mosher, inconsequential. I’m quite happy to say that BEST is pretty much as good as we can get with the data we have, but even then is no more than a rough indicator of general trends. Has it warmed since the LIA? Yes. Has it warmed since c1980? Yes. Has there been no significant warming (or cooling) since c1998? Yes. Do we know any of these things within 0.1°C? Hell no.
But to claim there is deliberate fraud in the BEST adjustments, or that they constitute a massive scientific scandal is wrong. In my opinion, it’s the failure of the GCMs that Booker and Delingpole should be shouting about.
My main concern with homoginization is that the relationships that are used probably don’t hold over time.
I’m with Tony. BEST and the rest have produced a temp series that, whatever it reflects, it doesn’t reflect history. The Northwest passage was open and sailed in a single season by Henry Larsen in 1944. There was less ice in the Arctic in 1944 than now. It was obviously warmer in the Arctic then than now. We are told constantly that disappearing Arctic ice is an indicator of a warming world. You would not know that 1944 was hotter than now from BEST or the rest. I know BEST uses 16 data series, I know BEST uses raw data and only adjusts for breaks, measures against nearby stations when there are gaps and blah blah blah, whatever. BEST missed the 1940s heat. Something’s not right.
Steven, I read the posts linked at the top and never saw a reference to BEST or Berkeley Earth.
Did I miss it or is this a strawman?
Steve,
Very clean and to the point. Good job of showing the effort and direction that you are going. Thanks
“You wont like the answer if that approach is taken with…”
Lemme do my best Joshua here:
“They did it too, mommy!”
Andrew
There is no such a thing as ”raw data” all data has being cooked!: https://globalwarmingdenier.wordpress.com/
Unless, of course, you use the raw data like we do… http://berkeleyearth.org/source-files
So you really do the transcription from the paper records and QC yourselves?? Then we can blame all the garbage on you.
No, but we use the raw data directly transcribed from paper records and archived by folks. You do know that sources like GCOS, GSOD, GHCN-Daily, WMSSC, and others only report raw data, and ones that have homogenized data (e.g. GHCN-Monthly and CRUTemp) also have corresponding raw files.
Zeke Hausfather, I went to your link; that’s for confusing people, not to inform! Here are the real proofs / facts; ”all proven beyond any reasonable doubt” :: https://globalwarmingdenier.wordpress.com/2014/07/12/cooling-earth/
stefan –
Nice, bro. Keep up the good work!
Stefan,
You might like this resource: http://denialdepot.blogspot.com/
(I’m really hoping you are a very convincing Poe).
Zeke Hausfather
correction: you Warmist are Poes, BUT not very good Poe – I stand only for the truth; All can be proven, now! Zeke, I read your link, read my post!
BOTH EMPERORS ARE NAKED!!!
What about the lost data?
http://rogerpielkejr.blogspot.com/2009/08/we-lost-original-data.html
Zeke – Would you please clarify the approximate portions of raw vs. previously-adjusted data you used?
“Whenever possible, we have used raw data rather than previously homogenized or edited data.”
source: http://berkeleyearth.org/about-data-set
Zeke Hausfatehr said “Unless, of course, you use the raw data like we do…”
Zeke, hasn’t Mosher insisted there is no such thing as raw data?
You know, the way Mosher argues that white is black and black is white?
You know Mosher.
No such thing as raw data.
We live in a strange world with an increasingly clear battle-line separating:
1. Political powers and consensus ‘scientists’ adjusting experimental data on one side.
2. A benevolent creator and sustainer of every atom, life and world on the other side.
Side 1 directs society to UN’s Agenda 21: http://habitat.igc.org/agenda21/index.htm
Side 2 directs society to a much Higher Power: https://dl.dropboxusercontent.com/u/10640850/Solar_Energy_For_Review.pdf
@Zeke Hausfather I went to your link; that’s for creating confusion – here is the truth and all the facts::: the ”correct” monitoring is completely WRONG, not only the manipulated data; therefore: the overall ‘’global’’ temp is same every year, BUT hypothetically: even if there was any fluctuation in temp, nobody would have known, because nobody is monitoring on every 10m3, for every minute in 24h!!!
1] monitoring only for the hottest minute in 24h and ignoring the other 1439 minutes, in which the temp doesn’t go up, or down, as in the ”hottest” minute…. statistically 1439 minutes against one…?!?! Hello ”statisticians! It’s same as: if the car is got 1440 different parts, but you are building that car, ”with one bolt only” you will not get very far…! Some places, sometimes warms by 5C from midnight to midday – other places at different times from midnight to the hottest minute in the day – IT WARMS UP BY 20c-25c- and 30C, in 12h difference – no swindler takes those things into account! Why?! Therefore: ”the hottest minute in 24h misleads by at least 10C!!! They conveniently overlook that; then ”pretend” to know to a hundredth of a degree precision, for the whole year, for the whole planet?!?! The ”Skeptics” are getting fattened on the Warmist bullshit…
2] the ”highest temp minute in 24h, is not at the same time every day! Sometime is the ”hottest at 11, 50AM, most of the time is after 1pm = that is many more warmer minutes than previous day.
3]example: tomorrow will be warmer by 2C than today, happens many times; and they will record 2C warmer – because the ”warmest” minute was 2C warmer, not ALL the rest of the 1439 minutes since midnight were warmer by 2C. b] question is: ”is it going to start from midnight, every minute to be warmer than the same minutes in previous day?! Therefore: recording only the hottest minute is meaningless! Nobody knows what was the temp yesterday, or last year on the WHOLE planet… but most of the fanatics in the blogosphere pretend to know with precision the temp for the whole year, for last thousands of years… What repetition and aggressive propaganda can do to a grown up person’s brains… tragic, tragic…
4] on a small hill, put a thermometer on all 4 sides; all 4 will show different temperatures on different day and on SAME minute simultaneously – when you take in the account that: on many places one thermometer represents millions of square kilometers, where are thousandths of ”independent” variations, every few minutes, on every different altitudes = gives a clear picture about their ”global temperature” for last 50years… or 5000 years, or for the last two years. On small part of the planet is warmer for few weeks than normal – they declare it as: ”warmer year”… what a science… couple of months after, when on that same place is colder than normal – they avoid that place and point other place where is for 3-4 days warmer than normal… what a brainwashing scam…
5] pointing at some place that is warmer than normal – is SAME as saying: ”the planet is warmer by 12C at lunch time, than before sunrise…? taking in consideration the size of the planet: one thermometer or 6000 thermometers, wouldn’t make any difference! ( look at their ”global” temp charts… they look like seismographs… with ”precision” to one hundredth of a degree, for the last thousandths of years… = the biggest con /lies since the homo-erectus invented language…
6] a thermometer can monitor the temp in a room; but one thermometer for 10 000km2?!
7] even those ”few” (6000) thermometers are not evenly distributed; no honest statistician would have taken to make ”statistic” if he wasn’t told: which individual thermometer, how much area represents. Example: if the workers in 4 countries have their pay packet increased by a dollar, and in 2 countries had ”decreased by a dollar Q: would the ”overall’ all workers in those 6 countries get more money, or less? Of course, statistic would say: ‘’yes’’ (the 4 countries were Luxembourg, Monaco, Belgium and Portugal, increased by a dollar. The other two were India and Chinese workers, decreased by a dollar) statistic would be wrong; because two thermometers represent much larger area than the other four combined. So much about the ‘’correct’’ temp data… (there are more thermometers in England monitoring for IPCC, than in Russia… England is a small dot on the map (but most of the lies come from there) Warmist Poms are the most shameless liars..
8] when is sunny – on the ground is warmer / in upper atmosphere is cooler – BUT, when is cloudy, upper atmosphere is warmer, on the ground cooler – overall same temp; BUT, because ALL thermometers monitoring are on the first 2m from the ground = they are completely misleading! There is much less heat in the first 2m from the ground, than in the rest of 10km up. The rest of 10km up, is not on their ”globe”…?!
9] for the shonks northern hemisphere summer is warmer by 3,8C than S/H summer. That tops the stupidity; they can’t get it correct even about same year. They come WRONG by 3,8C for two different reasons: a] N/H has more deserts, southern hemisphere has more water. Desert has warmer top temperature, BUT the night temperatures are cooler – by not taking all minutes in 24h, they are wrong by +/- 3C. In deserts get to 45-50C at day time, but nights are cold -/ on islands in south pacific between day / night temp is different 3-5C, is that science? B] on southern hemisphere are ”LESS” thermometers = less thermometers cannot say correct temperature against the N/H more thermometers, when you summon up all the numbers. So: only by those two factors they are wrong by +/- 3C, but when you say the last year’s temp cooler by 0,28C than today’s = it shows the sick propaganda… they call themselves ”scientist” Instead going to Antarctic, Arctic to get reumatizam and spend millions, they can get the whole truth on my blog; but they are scared from the truth as the devil from the cross… The truth: if they have same number of thermometers, distributed evenly AND every minute in 24h is taken in consideration = would have shown that: every day and every month of every year and millennium is ”overall” same temperature on the earth!!!
10]almost all of those 6000 thermometers collecting data for the ”climatologist; are distributed on land – water covers 2/3 of the planet!!! If you don’t understand what that means… you are qualified to be a ”climatologist”…
11]When you point out to them that: ‘’6000 thermometers cannot monitor the temp in the whole troposphere – thermometer is good to monitor room temp, but not one thermometer for 1000 km2 – 6000 thermometers is not enough to monitor the temp in all Hilton Hotel’s rooms’’ -> they instantly point out that: ‘’there is satellite temp monitoring’’! Well, ‘’satellite’’ is a great technology, very impressive; unfortunately, they don’t have 350km long thermometers, to monitor from space the temp on the ground! They use infrared photos that never covers the whole planet, in two color blotches for the whole of Pacific, or for the whole of US. The ‘’two’ colors represent THE different temp, BUT: if you look the evening weather report, it says that are many variations in temp even for the big cities in USA; would be much more variations if they were reporting for every square mile! B] temp distribution is three dimensional in the atmosphere and constantly changes, cannot present it on two-dimensional picture! Satellite monitoring is the biggest con! Unfortunately, person responsible for analyzing those pictures will not admit the truth – because he prefers to be seen as very important, by the gullible foot-solders, the lower genera and IQ Warmist & Skeptics…
12]Earth’s temperature is not same as temperature in a human body I.e: if under the armpit goes 0,5C up = the whole body is higher by 0,5C, arms, legs, the lot. Therefore, can tell if is gone higher or not. Earth’s temp is different on every 100m and FLUCTUATES ”INDEPENDENTLY”! Which means: one thermometer cannot tell the temp correctly for 1km2, when one monitors for thousands of square kilometers = Warmist can get away with that sick trick, thanks to the ignorant phony Skeptics and bias media… (Skeptics don’t need even thermometers, a pit-bog on each hemisphere is sufficient for them… Nobody knows what was the earth’s temp last year – to save his / her life! They ”pretend” to know what was the earth’s temp for thousandths of years = that’s Warmist & Skeptic’s honesty!
Using only those 12 points above; to put any leading Warmist on a witness stand, under oath => will end up in jail; for using the ”temperature data / charts” as factual =/ Skeptics & Warmist of lower genera and IQ in a nuthouse, for believing in warmer / colder years. Warmist only prosper and flourish, thanks to the Skeptic’s outdated Pagan beliefs…https://globalwarmingdenier.wordpress.com/2014/07/12/cooling-earth/
Wow. Just… wow.
Rud Istvan wrote in previous post | February 7, 2015 at 4:18 pm |
“Steven, no quibble with any of your points except the last. Perhaps my meaning was not clear. It is indisputable that for at least a decade, the past has been cooled and, in some cases, the present warmed compared to ‘raw’. That is so for USHCN, GHCN, BOM Acorn, NIWA… Moreover, the tendency has been increasing, documented by simple archival comparisions of regional and global temperature series at different points in time. For example, Between 1999 and 2011 NOAA NCDC cooled CONUS 1933 and warmed 1999 by a net total of 0.7C and erasing the mid 1930s heat record. It is also indisputable that all this opposite to what NASA GiSS says it does in homogenization to resolve UHI, using Tokyo as the illustration. It is also indisputable that GHCN homogenization has with statistical ‘certainty’ a warming bias. See the Europe AGU 2012 paper available at http://www.itia.ntua.gr/en/docinfo/1212. All examples and more in essay When Data Isn’t, with footnotes”
Rud tells half the story. you’ll have to ask him why he and other ignore the following facts.
1. Berkeley COOLS the global trends since 1960, the period most relavant for attribution studies
2. berkeley COOLS the global trends from 1979 to present.
3. We cool large portions of the largest continent. Africa
4. The AGU paper looked at 181 stations. BAD skeptic.
and more,
However if you like raw data, we can produce a raw ocean and raw land.
Trends will INCREASE.. big time.
I like raw data,
Real Raw data.
Like you get on land for 130 years.
Just show the land real raw data Steven.
And discuss it.
No one measuring raw data on or in the sea 130 years ago of any significance at all [note ‘of any significance”]
Hence your attempt to inject sea water into the debate is a big red herring, misleading , fallacious, as quoting “Zeke Hausfather | February 9, 2015 ” ocean temperature adjustments significantly increase temperatures prior to 1950.”
Amazing!
reductio ad absurdum.
” The net effect of all adjustments is to notably reduce the century-scale warming trend compared to raw data:”
Or in plain English [sorry, American] We can hide the way we dropped the past land temperatures.
By putting in made up sea ones.
Yup, if you didn’t artificially cool that “RAW” data someone might smell something!!
“Berkeley COOLS the global trends since 1960”
So does GHCN. From here is this graph of trends from x-axis year to present. Adjusted purple, unadjusted blue.
http://www.moyhu.org.s3.amazonaws.com/pics/trendback.png
The EAGU paper looked at ALL ghcn stations with reasonably complete 100 year records. That was the sole selection criterion for the sample. For example, De Bilt Netherlands, home of KNMI, with a well maintained station that may have some UHI (for which the NASA GISS recommended adjustment is to warm the past to compensate). The past was cooled, instead.
angech2014 | February 10, 2015 at 12:02 am |
“Or in plain English [sorry, American] We can hide the way we dropped the past land temperatures.
By putting in made up sea ones.”
In the mode of Wigley/Jones:
“Phil, Here are some speculations on correcting SSTs to partly explain the 1940s warming blip. If you look at the attached plot you will see that the land also shows the 1940s blip (as I’m sure you know). So, if we could reduce the ocean blip by, say, 0.15 degC, then this would be significant for the global mean — but we’d still have to explain the land blip. I’ve chosen 0.15 here deliberately. This still leaves an ocean blip, and i think one needs to have some form of ocean blip to explain the land blip (via either some common forcing, or ocean forcing land, or vice versa, or all of these). When you look at other blips, the land blips are 1.5 to 2 times (roughly) the ocean blips — higher sensitivity plus thermal inertia effects. My 0.15 adjustment leaves things consistent with this, so you can see where I am coming from. Removing ENSO does not affect this. It would be good to remove at least part of the 1940s blip, but we are still left with “why the blip”. “
That’s something that makes me wonder. Its clear that NOAA and the other main datasets are applying a positive bias. BEST does things differently and gets relatively similar results. But I’m still left with that nagging feeling that the underlying problem is that the majority of maintenance of the surface temperature network…results in steady warming followed by a break point to cooler temperatures.
I am quite open to the idea that BEST is correct. But I also know that the maintenance of the surface network was NEVER intended to provide the sort of data we’re using. Literally, no matter how you slice it….as long as you try to stick it together into an unbroken record OF ANY KIND, you’re going to have to deal with an upward slope….which they might be inadvertently sticking end to end without correcting the bias. And that’s on top of the fact that the world most likely has been warming.
I was trying to explain this using a thought experiment…of using three POTENTIAL locations for stations that all (for this example) had the same temperature, the first being the initial location near a city. The next two farther and farther out. As UHI became a problem the station would be moved from the first to the second. Then as UHI became a problem again it would be moved from the second to the third. If you try to stitch that “single station” record together, it will look like there’s warming. Throw in multiple neighbors going through the same process and while yes, you can find the break points with greater certainty…you still can’t tell if the warming is UHI or the environmental.
HERE’S A THOUGHT. What if we make a pretend world that we KNOW the warming rate of as well as UHI contamination. Someone (I’m not up to the task) could create a world not unlike the one I mentioned. It would be a world where an increasing number of stations were subject to UHI contamination. ALL data would be known in advance…like the temperatures of the new sites stations would eventually be moved to. From that point on, its a relatively simple test. Use that input data and see what happens when it goes through the processing methods used by BEST, GISS, Hadley, etc.
Maybe they will add warming. Maybe they won’t. Maybe in some methods, the REAL warming will manifest partly (or entirely) as “adjustment” (as reasonably suggested by Luboš Motl).
Point is ter change it… BOM ACORN..
http://en.wikipedia.org/wiki/Police_box#mediaviewer/File:TARDIS2.jpg
The problem with Vaughhan Pratt comment above “you use the “available” data is that the “available” data was altered. As stated in a prior comment, if the data is so rotten it needs to be altered it calls into question to validity of said data. Ergo it is immoral to base energy policy that impacts millions of lives on data that needs to be altered even by the most honest people. The problem is even if there is no intent to be dishonest people who are convinced that CO2 is the primary culprit behind climate change will have what some have referred to as “Confirmation Bias” Confirmation may explain why so much of the raw data has been altered to cool the past and warm the present It’s human nature.
Confirmation [bias] may explain why so much of the raw data has been altered to cool the past and warm the present. It’s human nature.
You on the other hand are immune to confirmation bias. The world eagerly awaits your secret.
Pingback: Did NASA and NOAA dramatically alter US climate history to exaggerate global warming? | The Fabius Maximus website
This post says:
But if you look at just the blue and red lines, you can see a .2 to .3 degree difference in the earlier portions. That means there’s a 15-20% change introduced by these adjustments. And that’s for if we only go back to 1850. The BEST temperature record extends another hundred or so years back. We can’t see how much of a difference its adjustments make in that period.
It’s also not clear to me the timing of the adjustments is as meaningful as this post makes it out to be. BEST estimates its climate field over a particular period (just which period is used has been a source of some confusion). It is possible the adjustments have a greater effect in the past, at least partially, because they are further from the baseline period.
This is simply untrue. We don’t need to know “the true field perfectly” to decide whether or not the resolution of a temperature record is too coarse. We don’t need to know “the true field perfectly” to compare the spatial resolution of BEST’s results to that of other groups’ results and judge which are more useful.
We can do far more than “note [BEST] lacks the edge of gridded approaches and tends to have a smoother field.” This post tries to hand-wave away the issue, but We can specify how much smoother BEST’s temperature field is and make judgements about whether or not it’s a bad thing. In fact, we should.
On a final note, I have to say comments like this are strange to me:
Because if I read this without seeing the graphs, I’d never guess “very little” means several tenths of a degree. I don’t consider 15% or more to be “very little.” I’m sure there are other people who don’t as well. I imagine there are also people who do. It’s a judgment call, and there is no “right” answer.
It is quite unhelpful to simply dismiss something as “minor” when people can reasonably believe is it not.
Brandon,
The impact since 1900 is on the order of 0.05 C, not particularly large. The magnitude of this adjustment is effectively the same if you use metadata + empirical breakpoints or metadata alone (see Figure 4).
The impact globally, that is. Specific regions have larger adjustments, as discussed in the piece. For the U.S., at least, we have good evidence for large systemic biases (TOBS, MMTS, etc.): http://judithcurry.com/2014/07/07/understanding-adjustments-to-temperature-data/
Zeke Hausfather, I have no idea why you say this. I don’t disagree, but how does it relate to anything I said?
last month
0.05 degrees was measurable and important
now it’s insignificant
this game is hard to follow
“The impact since 1900 is on the order of 0.05 C, not particularly large. ”
Then the obvious question is: why continue to make the adjustments?
Sure, run a test to make sure the adjustment is still negligible when new data arrives, then just report that and use the raw data. If someone comes up with a detection scheme that shows it is significant, report on that.
Zeke, I went to your earlier Climate Etc. link (2014/07/07, linked in this thread nearby) and it shows that the NCDC raw data, global, around 1900 through about 1925 is about 2 tenths of a degree warmer than the NCDC homogenized data (5 year smooth). By around 1950, the homogenized and raw are nearly equivalent. For the US, the differences are larger for longer and don’t peter out until about 1990.
I may be missing something, but your response to Brandon says the magnitude of adjustments is on the order of 0.05 degrees. That is much less than the differences between raw and homogenized data in your 07/07 post, linked nearby in this thread.
What am I missing? Are the adjustments you are talking about not the differences between raw and homogenized data? And if they are not, why are there differences of up to 2 tenths of a degree, for several decades, between the raw and homogenized data?
I’m not on the conspiracy bandwagon, just looking for understanding.
OK, I think I see the source of my confusion. Rereading Zeke’s 07/07 post about adjustments, that post is about the several reasons why raw data from the past had to be adjusted. Things like changing the time of day at which the temps are read from PM to AM, resulting in an artificial cooling trend, if not adjusted. THOSE adjustments are what causes adjusted data, globally, to be about 2 tenths of a degree cooler than raw data in the early 1900s, with the difference between the two sets petering out around 1950. Bigger differences, lasting longer, for the US, in that post of Zeke’s.
So why does Zeke say that the impact of adjustments since 1900 is tiny, on the order of 0.05 C? This is the source of my confusion, that led me to post earlier.
Here is a guess: when Zeke uses the word adjustment in his reply to Brandon (Feb. 9 at 11:58), is he referring to further adjustments, AFTER the basic adjustments to the raw data described in his 07/07 post? If this is the case, the use of the word adjustments in both cases threw me off.
Zeke, can you please confirm that this is what happened?
“The impact since 1900 is on the order of 0.05 C”
Larger than enough for the Director of NASA to broadcast a record in hottest year for the planet….
First things First
In this post
http://wattsupwiththat.com/2015/01/29/best-practices-increase-uncertainty-levels-in-their-climate-data/
you messed up, in particular not noting the change in spatial uncertainty.
I would like to see you add a correction to that post, and an acknowledgement.
Now here>
“But if you look at just the blue and red lines, you can see a .2 to .3 degree difference in the earlier portions. That means there’s a 15-20% change introduced by these adjustments. And that’s for if we only go back to 1850. The BEST temperature record extends another hundred or so years back. We can’t see how much of a difference its adjustments make in that period.”
The change is on the order ( as Zeke notes ) of about .05C. since 1900.
Since 1960 we reduce the trend. Since 1979 we reduce the trend.
These changes and the changes before are inconsequential to any interesting question in the climate science debate. They don’t effect our core understanding of A) sensitivity. B) attribution. C) reconstructions
D) GCM performance. In short, they may be of minor technical interest,
but in terms of interesting science.. they are just pedestrian.
##############################################
‘It’s also not clear to me the timing of the adjustments is as meaningful as this post makes it out to be. BEST estimates its climate field over a particular period (just which period is used has been a source of some confusion). It is possible the adjustments have a greater effect in the past, at least partially, because they are further from the baseline period.”
WRONG. we used a 100 baseline period here. We studied the baseline period as I told you. It makes no difference.
“This is simply untrue. We don’t need to know “the true field perfectly” to decide whether or not the resolution of a temperature record is too coarse. We don’t need to know “the true field perfectly” to compare the spatial resolution of BEST’s results to that of other groups’ results and judge which are more useful.”
wrong. too smooth means smoother than the TRUE FIELD.
#######################################
“It is quite unhelpful to simply dismiss something as “minor” when people can reasonably believe is it not.”
Here is what you have to do to prove that its CONSEQUENTIAL.
show that the adjustments CHANGE some core science in a meaningful way.
1) Show, that is DO THE WORK, to demonstrate that our belief that
over 50% of temperature change since 1950 is due to man.
Attribution is CORE claim of AGW.
2. Show that adjustments have a meaningful effect on estimate of sensitivity such that the consensus position of 1.5C-4.5 is overturned
3. Show that adjustments have a meaningful effect on climate reconstructions. for example the HS
4. Show that adjustments have a meaningful effect on climate forecasts made by GCMS.
Changing the global series by small amounts, heck even large amounts, have very little leverage into the key uncertainties in climate science. Of course it makes a good blog post, but not impactful science.
Now you can prove that “reasonable” people are right to consider these differences as “major” publish a paper showing how the core science is challenged.
Ever heard the phrase polishing a bowling ball?
Steven Mosher, it’s obnoxious to say things like:
When I directly addressed this point less than two hours after you first brought it up. Ignoring people’s responses to what you say then repeating yourself is just rude.
This in no way addresses anything I said. This post talks about the effects of BEST’s homogenization, not just the effects of BEST’s homogenization after 1900.
You never actually told me this.
Also, what you say does nothing to demonstrate what I said is wrong. You say you used a 100 year baseline, but that doesn’t contradict what I said. In fact, it makes what you and Zeke say incredibly peculiar. You both make the point the effects of homogenization are small after 1900, but if you used a 100 year period, that means the non-baseline period ends ~1900.
This does nothing to indicate I am actually wrong. This post refers to knowing “the true field perfectly.” I specifically said we don’t need to know “the true field perfectly.” We can make judgements about whether or not BEST’s results are too smooth without knowing “the true field perfectly.”
I said adjustments to the BEST results shouldn’t be described as “minor” or “very little” because people may reasonably misinterpret what is meant by those words. Your response does nothing to rebut that. Instead, it seems to advance a crazy position, that it is acceptable to call any adjust “minor” if one can’t show the adjustment changes “some core science in a meaningful way.” Indeed, you say:
Even though I clearly discussed the effect BEST’s adjustments have on BEST’s results. Determining whether or not adjustments have “very little” effect on BEST’s results does not require determining whether or not adjustments to BEST’s results somehow change our understanding of climate science.
This is easily demonstrated by applying your argument to any other piece of work. Thousands of papers are published every year which don’t speak to “the core science” of global warming. Suppose one of these papers got results which were 99% due to adjustments. Your response would suggest it is fine to call those adjustments “minor” because anyone who disagrees needs to show “how the core science is challenged” by the existence of those adjustments.
This entire response of yours is a mess.
Brandon
“When I directly addressed this point less than two hours after you first brought it up. Ignoring people’s responses to what you say then repeating yourself is just rude”
No. you need to post a correction in your post.
probably my Last response to you until you do that
Steven Mosher:
I did. You’ve conveniently ignored that.
You mean you’ll stop making things up in obvious ways to insult me and dismiss the valid points I make? Alright. Running away would be a better plan than what is practically lying about your critics.
Stephen Mosher: Quotes Branson Shollenberger“But if you look at just the blue and red lines, you can see a .2 to .3 degree difference in the earlier portions. That means there’s a 15-20% change introduced by these adjustments. And that’s for if we only go back to 1850. The BEST temperature record extends another hundred or so years back. We can’t see how much of a difference its adjustments make in that period.”
Mosher’s response: The change is on the order ( as Zeke notes ) of about .05C. since 1900.
That was a good point, and the whole post was a good post.
Brandon Shollenberger: We can make judgements about whether or not BEST’s results are too smooth without knowing “the true field perfectly.” :
No you can’t. All you can show without knowing the true field perfectly is that one smoother is more or less smooth than another (or no smooth at all.)
Matthew R Marler:
Since you guys are using the extreme of “knowing the true field perfectly,” let’s take this to the extreme and suppose we smoothed the temperature data so much the value was the same across the entire globe. That would be indistinguishable from simply using the global temperature record at every part of the globe.
According to you guys, we can’t say that is bad. According to you guys, we need to know “the true field perfectly” to be able to make judgments like, “There should be some amount of spatial variation.” Just about everyone can agree temperatures in Antarctica are changing at a different rate than temperatures at the equator. If I saw results smoothed so much they found the same rate of change in Antarctica as at the equator, I’d say that is “too smooth.”
According to you guys, I’d be wrong. According to you guys, if we don’t know “the true field perfectly,” we can’t make any judgments based upon it at all. I’m sorry, but this is just ridiculous. You guys wouldn’t accept it on any other issue. You guys would scoff at anyone who said we need to have perfect information in order to draw any conclusions.
Brandon Shollenberger: let’s take this to the extreme and suppose we smoothed the temperature data so much the value was the same across the entire globe.
That does not help you to determine whether BEST smoothed the data too much.
Matthew R Marler:
I find simply asserting someone is wrong tends not to accomplish much. Would you mind explaining why I am (supposedly) wrong rather than just saying I am? If you don’t, I’m going to just wind up responding, “Yes, it does.”
And then you’ll be all, “Nuh-uh.”
And I’ll be all, “Yuh-huh.”
And we’ll keep going back and forth until one of us realizes it is a terrible way to participate in discussions that just makes anyone doing it look obnoxious.
Brandon Shollenberger: I find simply asserting someone is wrong tends not to accomplish much.
“Taking it to extremes” is counterfactual — they did not do that. They have described what they did and shown some outputs. We can not tell without knowing the exact values whether they have “over smoothed” or not.
Another example of “oversmoothing” would be to take a single number, the calculated mean or calculated conditional global expected value, as representative of the entire Earth; calling it an “equilibrium” temperature; and then using that as the basis for calculating a new “equilibrium” temperature that would result from doubling the CO2 concentration. The BEST team have not done that either.
Matthew R Marler:
Uh… duh? Of course they didn’t do it. The point of reductio ad absurdum arguments is to show what happens if you apply an argument to a more extreme situation so as to indicate why that argument fails to work in a less extreme situation.
The entire point of me saying I was taking the argument to the extreme (which you misquoted) was to make it clear I wasn’t referring to what BEST actually did.
The sentence before this doesn’t do anything to support this, meaning you’ve again offered no basis for this claim.
But according to you and BEST, we couldn’t say this is over-smoothed because we don’t have perfect information. BEST could literally claim every location has warmed at the exact same rate, and according to you guys, we couldn’t say they’ve over-smoothed their data. We’d have to just shrug our shoulders and say, “Eh, maybe they got the spatial resolution right.”
This is incredibly simple. I say we can make judgments about whether BEST’s spatial resolution is better or worse than that of other groups by using the information we have. You guys say we can’t do that because the information isn’t perfect. That’s all there is to it.
> The point of reductio ad absurdum arguments is to show what happens if you apply an argument to a more extreme situation so as to indicate why that argument fails to work in a less extreme situation.
Perhaps not:
http://www.iep.utm.edu/reductio/
Brandon Shollenberger: The point of reductio ad absurdum arguments is to show what happens if you apply an argument to a more extreme situation so as to indicate why that argument fails to work in a less extreme situation.
Yes. That is the point. I would call it a “claim”. But the extreme is so different from the actual case in this instance, that you can’t show how it invalidates the original assertion. You can not show that the BEST team over smoothed without knowing the exact data, but it is oversmoothing to use one value for a large area. Where in between you pass from requiring exact data to requiring less exact data to requiring no data at all would require knowledge of the specific cases.
Reductio ad absurdum only works in mathematics, not in empirical science where all statements of knowledge are only approximate, and the absurdity only refers to the extreme, not the topic of focus. Clearly, if you consume a pound of salt per day you’ll get sick, but lightly salting your salad is no harm whatsoever. As has frequently been pointed out, reductio ad absurdum (taking the extreme case) is of no use whatsoever in toxicity studies. One of the substances that you do not want too much of is oxygen.
Again, you’ve offered no basis for your position. Instead, you’ve seemingly admitted it is wrong:
I can’t show BEST is oversmoothing, but I can show using one value for a large area is oversmoothing… and I have shown BEST uses one value for a large area.
But that doesn’t prove BEST is over-smoothing because it is BEST, and apparently that means the problem you acknowledge can be shown can’t be shown because… they wouldn’t be the best? Is that the argument? I don’t know. I can’t tell. You’ve still done nothing to explain why we can’t say it is bad that BEST shows practically the same results for entire continents.
Brandon Shollenberger, I can’t show BEST is oversmoothing, but I can show using one value for a large area is oversmoothing… and I have shown BEST uses one value for a large area.
This is where you began: We don’t need to know “the true field perfectly” to decide whether or not the resolution of a temperature record is too coarse. We don’t need to know “the true field perfectly” to compare the spatial resolution of BEST’s results to that of other groups’ results and judge which are more useful.
We can do far more than “note [BEST] lacks the edge of gridded approaches and tends to have a smoother field.” This post tries to hand-wave away the issue, but We can specify how much smoother BEST’s temperature field is and make judgements about whether or not it’s a bad thing. In fact, we should.
Rereading your posts, I do not find where you ever showed that BEST was oversmoothing by using one value for a large area. Your comment was about comparing different smoothing algorithms (or different smoothed outcomes) and deciding which of them did the most “useful” smoothing. But unless you know the true values, you can not tell which algorithm was the most accurate. Without knowing which one is most accurate, you would be hard-pressed to tell which would be most “useful”.
All you have shown is that in some other circumstance, you might be able to do something other than what you started out describing here.
Matthew R Marler:
Then you haven’t read my posts on this topic. I’m guessing you meant comments, not posts. If so, the reason you haven’t seen me show this in my comments here is it seems silly to show what BEST clearly shows in its post. What point would there be in me replicating what is clearly shown in Figures 5, 6, 7 and 8?
If you had absolutely no idea what the true values are, you’d be hard-pressed to tell which algorithm is the most accurate. If you had a good, but not perfect, idea what the true values are, but it wouldn’t be so difficult to tell which algorithm is most accurate.
Apply your argument to any other topic, and you’ll see it is clearly ridiculous. In no other topic would you say we need perfect information to make any judgments. If you did, you’d wind up saying things like, “We can’t tell if the planet has warmed because we don’t have perfect information about what the planet’s temperatures are.”
We don’t need perfect information to decide whether or not BEST over-smooths its data. We just need useful information, of which we have plenty.
This is precisous
“But if you look at just the blue and red lines, you can see a .2 to .3 degree difference in the earlier portions. ”
Red is Africa
Blue is the US
when we talk about the adjustments being inconsequential GLOBALLY
we mean the BLACK line
The red line is Africa– 20% of the land
the blue line is the US 5% of the land.
So yes if you look at 5% of the data ( blue) you see a .2 to .3 degree difference
the POINT of showing people continents and how they differ is so that people will AVOID the kind of mistake Brandon just made.
GLOBALLY ( we are estimating the GLOBAL average) the adjustments are mousenuts..
BUT because people can cherry pick ( the US) they can show BIG differences.. BUT they also IGNORE big differences in the other direction.
Steven Mosher, are you trying your hardest to misunderstand me? You say:
Yet this is obviously not what I did. The quote you provide from me was provided in direct response to:
Which I quoted. I clearly and specifically discussed Figure 1. Figure 1 does not have lines for Africa or the United States. The figure with the lines you refer to is Figure 4, described as:
You just wrote 150 words to mock me entirely by ignoring what I was responding to and coming up with a completely ridiculous interpretation of what I said nobody with basic reading skills could innocently come up with.
Seriously, are you intentionally trolling me?
Steven Mosher,
You wrote –
“GLOBALLY ( we are estimating the GLOBAL average) the adjustments are mousenuts..”
Why bother – or are you only kidding, and mousenuts means either vaishingly small, or quite significant. Ah, the benefits of Warmist scientism!
What is the Warmist definition of mousenuts? Do you find them locally, or only GLOBALLY? Are GLOBAL mousenuts bigger than local mousenuts?
Have you the faintest idea what you are talking about? Does anybody else care?
Live well and prosper,
Mike Flynn.
Brandon Shollenberger: Which I quoted. I clearly and specifically discussed Figure 1. Figure 1 does not have lines for Africa or the United States.
Tough luck Steven Mosher, you goofed on that one.
Matthew R Marler:
Good luck ever getting him to admit this.
It’s pretty pathetic BEST is having this sort of behavior be part of its public outreach. “You criticized us? We’ll completely misrepresent what you say in obvious ways to insult you! Because that’s how we deal with legitimate concerns!”
Brandon Shollenberger: “You criticized us? We’ll completely misrepresent what you say in obvious ways to insult you! Because that’s how we deal with legitimate concerns!”
You are overwrought. It was a simple goof.
You know that Brandon is very sensitive, Matt. He’ll calm down in about a week.
Matthew R Marler:
You might consider the possibility you aren’t as good at reading people’s minds/tones as you might think. I’m not overwrought at all. I think this is hilarious. That’s why I gave an exaggerated impersonation, complete with exclamation marks. It’s a common and humorous form of mocking people.
Overwrought would be more like if I started resorting to using Caps Lock like:
Or:
But I don’t feel any need to do things like that, just like I don’t feel a need to use bombastic rhetoric or derogatory remarks. I don’t know what makes you think I am overwrought, but I am not. The most I am is sardonic.
Or maybe I am overwrought. Maybe I am and I just don’t know it. Maybe I need you to tell me what I am feeling so I can actually know!
Brandon Shollenberger: The most I am is sardonic.
OK, I’ll abandon the inference about your mood and stick with my judgment about your response to Mosher: you overreacted.
I wrote a snarky reply to your claim to be at most sardonic, including a lot of quotes from your replies. Thankfully, it has not appeared. I must have goofed, but it worked out better this way.
Stages:1 denial, 2 anger, 3 bargaining, 4 depression, 5 acceptance.
1. Denial. Yet this is obviously not what I did.
2. Anger. You just wrote 150 words to mock me entirely by ignoring what I was responding to and coming up with a completely ridiculous interpretation of what I said nobody with basic reading skills could innocently come up with. Seriously, are you intentionally trolling me?
3. Bargaining. Skipped that stage.
4. Depression. The most I am is sardonic.
5: Acceptance. Or maybe I am overwrought. Maybe I am and I just don’t know it. Maybe I need you to tell me what I am feeling so I can actually know!
Brandon Shollenberger: Quotes Steven Mosher: As Figure 1 illustrates the effect of adjustments on the global time series are tiny in the period after 1900 and small in the period before 1900. Our approach has a legacy that goes back to the work of John Christy when he worked as state climatologist: Here he describes his technique.
Brandon Shollenberger: But if you look at just the blue and red lines, you can see a .2 to .3 degree difference in the earlier portions. That means there’s a 15-20% change introduced by these adjustments. And that’s for if we only go back to 1850.
There are a few adjustments as large as 0.2 degrees, but they have little effect on the estimates of the global trends. converting 0.2 – 0.3 to 15%-20% does little to add to your point: what is the base?
Matthew R Marler:
Suppose I am correct that the timing of the effect of the homogenization algorithm is (at least in part) an artifact of their choice of baseline period when calculating their climatology fields. If I am, it would mean BEST could have its modern temperatures change by as much as several tenths of a degree just by using a different baseline period. I think that’s something people might care about, even if that didn’t affect global trends (and I have no particular reason to think it wouldn’t).
This is especially true given the fact BEST released a report for the media discussing whether or not 2014 was the hottest year, a report which gave significant attention to differences less than a tenth of a degree. BEST apparently cares enough about differences of ~1% to release a report on it to get media attention. I don’t see how it can do that then dismiss concerns which affect their results by more than 10 times as much.
I’m not sure what you mean when you say “base” here. I was tempted to just be snarky and say, “10.” Instead, I’ll explain the reasoning. Looking at the BEST figure, I estimated it shows ~1.5C of warming since 1850. I then estimated that number would have been ~1.2C if they hadn’t calculated their “empirical breakpoints.”
I think it is useful for people to know ~20% of the warming shown in the figure is due to this particular homogenization step. Doing so lets them know the total amount of warming, the amount of warming caused by these adjustments and how the two relate to one another.
Brandon Shoilenberger: Instead, I’ll explain the reasoning. Looking at the BEST figure, I estimated it shows ~1.5C of warming since 1850. I then estimated that number would have been ~1.2C if they hadn’t calculated their “empirical breakpoints.”
That answers the question.
Its worth mentioning that while global land temperature adjustments tend to slightly increase the century-scale trend, mostly by reducing temperatures prior to 1950, ocean temperature adjustments significantly increase temperatures prior to 1950. The net effect of all adjustments is to notably reduce the century-scale warming trend compared to raw data:
https://pbs.twimg.com/media/B9cBLSACcAAyZ4_.png:large
Zeke, what’s the deal with the convergence of the raw and adjusted in about 1995 and an almost perfectly matched run-up to about 2002 and then a significant divergence for the remainder of the chart?
Its a result of ocean temp adjustments: https://twitter.com/hausfath/status/564939671247941632
Not sure of the reason for it, but I’m much less familiar with ocean adjustments in general. Ask John Kennedy.
Well, that is odd looking. In the early years the land and ocean temperature adjustments go in opposite directions. Then ocean temps don’t need adjusting for some time, then in 2002 they are adjusted up, from then on. Doesn’t look right. Very interesting charts, when you look at the land and ocean side by side.
I don’t see significant ocean temp adjustment for the 1995-2002 period I mentioned. Only after 2002. One would think the later data would need less adjusting than the earlier data. Looks weird.
Eyeball the charts and compare the land and ocean trends from 1910-1940. Why da ocean warming so fast? Weird. I am not believing they got good data.
It looks like the adjustments cooled the land and warmed the oceans, rather than climate physics having done it.
Zeke, you look taller and more intelligent than Mosher. Maybe Mosher needs glasses.
Hi Don,
The base period is 1961-1990, so the average is zero (or close to it) for both lines over that period. That often means that agreement looks better over the climatology period than at other times.
Since 2002, there has been an effort to increase the number of drifting buoys making SST measurements (note that drifters are not Argo floats). In 2005/06 the number of drifters increased markedly and their coverage improved too. Ships tend to measure warmer on average than drifters, so if you change the balance towards a network that’s more dominated by drifters, the result is an artificial cooling. You don’t see that cooling in either the ships or the drifters individually, nor do you see it in (arguably) the best available satellite record from ATSR instruments (see Figure 2-17 of Chapter 2 WG1 AR5). The HadSST3 adjustments account for the buoy-ship difference and I think that’s what you see in Zeke’s diagram.
Best regards,
John Kennedy
Diagramonkey, “The base period is 1961-1990, so the average is zero (or close to it) for both lines over that period. That often means that agreement looks better over the climatology period than at other times.”
Baseline choices for anomaly do have that feature/flaw.
https://lh3.googleusercontent.com/-VsVV9OrfMrg/VNn7zGllksI/AAAAAAAAMnQ/6AZpAra-WAs/w959-h549-no/temperature%2Bseries%2Bcorrelation.png
I was comparing “raw” as in absolute temperature estimates of Berkeley Earth and CRUT Ts with ERSSTv4 and HADIsst to see how much long range interpolation effected land temperature anomaly. The Berkeley data appears to use more coastal station data for “correcting” land station data than CRUT. Not sure if that is a good thing or a bad thing, just a thing.
Note: HADI from Climate Explore is in C degrees. Berkeley land from Climate Explorer needs the estimated monthly absolute value added to produce an absolute temperature data set. Both HADI and ERSSTv4 were in C degrees as provided by Climate Explorer.
If you try to construct a global actual temperature data set using Berkeley and ERSST or CRUT Ts and HADI you end up with something like this.
http://2.bp.blogspot.com/-3jfwPc_RldE/VNey7tky57I/AAAAAAAAMmU/E12drG_m98g/s1600/cmip5%2Ball%2Band%2Bmean%2Bwith%2Bobservations.png
Anomalies of course paint a slightly different picture.
Thanks John and Capt. I am still feeling the data prior to the 1940s is very sketchy. Maybe we should go to the treemometers for that.
Don
If asked I would say the global temperature record becomes reliable enough to use with the advent of Automatic weather stations in the 1980’s which should have some degree of correlation with satellite records always assuming the siting of the former is good.
I certainly wouldn’t bet the house on any sort of global or SST land record prior to this date. Individual country records may have a better provenance over a longer period but the number of reliable countries is relatively small.
tonyb
I certainly wouldn’t bet the house on any sort of global or SST land record prior to this date.
Unless you can quantify that, the house would be well advised not to let you.
“One final way to compare the results of various data producers is by comparing the spatial variability against that produced by global climate models. While not dispositive this comparison does indicate which temperature products are consistent with the variability found in simulations and those which are less consistent”
So.
The facts have to be consistent with the fiction??
This is cart before the horse stuff.
There is no reason on earth for facts to have to agree with simulations.
To state this is extremely unscientific.
The mucky statement ” not dispositive” meaning you claim [wrongly] the temperature record agrees with the simulations also needs correcting please.
.
It means exactly what it says. The comparison doesnt decide anything.
period. Its just a comparison.
Another way to look at this is as follows. If all the statistical models ( cru, giss, best, ncdc, ) had similar smoothness and the GCMs were different that would indicate that the GCMs might be too smooth. But, the comparison didnt turn out that way. So, we do the comparison. we dont hide the result. We show it.. conclusion? hmm doesnt show you anything that helps you decide.
Steven (and Zeke) – Thanks for your thankless work. I very much appreciate it – including but not limited to BEST, in the specific case of this post, and in the many (to the many power) thoughtful (and sarcastic [even sardonic!]) posts on this blog.
Even your noise is signal to my ears.
“Since the algorithm works to correct both positive and negative distortions it is possible to hunt through station data and find examples of adjustments that warm. It’s also possible to find stations that are cooled.”
If it did this properly the temperature could never change, could it.
But it does so the algorithm only works to correct some positive and negative distortions.
who gets to choose?
Cowtan and Way? taking out those 3 aberrant arctic stations that gave 40% “erroneous’ cooling.
Zeke when looking at those shorelines?
Some little clerk in the back office working for Greenpeace?
or some big clerk working for IPCC.
Let’s have a committee to vote on which ones to use .
Put Gavin in as chair.
“But it does so the algorithm only works to correct some positive and negative distortions.
who gets to choose?”
If a human chose things they you would be dubious of the human.
Here is what we did.
1. We built an algorithm. Note that we took a lot of ideas from skeptics
2. We tested that algorithm in a double blind fashion.
it corrected both positive and negative biases.
3. We applied that algorithm and reported the results.
4. We did sensitivity tests on the algorithm.
more break points, fewer breakpoints, no break points.
Guess what? We still cool the record in Africa, we still cool the globe since 1960, while.
Some people had a theory. Adjustments should even out. We tested that theory. It was wrong. Dr. Feynman tells you want to do.
This conspiracy ideation is ridiculous. Berkeley Earth was intended to give an independent measure of global temperature and it has done that. Skeptics may not like it, but that’s science. The temperature histories are quite robust.
I do think that there is a problem with the data, though, and it is a conceptual one.
A measurement is a measurement. It’s what the thermometer read on that day at that time. Measurements are raw data. Raw data should not be messed with.
Presenting “adjusted” measurements implies that the real data is not accurate and we know better now what it really was despite the measurement being, you know, raw data.
Generally speaking, in experimental science, the raw data is never changed. Analyses can model from it, by, say, using larger scales and adjacent measurements, but that in no way requires making adjustments to the original raw data. Smoothing raw data and treating the result as if it were raw data is a bad, bad, bad thing to do.
So the problem is that the climate science community, for some reason that I am not able to completely understand, has decided to replace their raw data with ever-changing “adjusted” data. The adjustments may make little or no difference to the big picture, but that’s not the issue. The issue is that now the community has decided that raw data is something that can be changed And the ability to change data ex post facto is a gigantic invitation to self-delusion and cherry-picking.
it’s probably too late to put the horse back into the barn, but I do wish that the community had chosen a more scientifically rigorous terminology and method to perform the various necessary operations on the raw data.
fizzymagic:
I agree, but I want to point out I’ve seen a number of people exhibiting no conspiracy ideation be dismissed as conspiracy theorists simply for raising concerns/criticisms. When people who behave properly get slimed and mistreated, it only encourages people to have take negative views.
BEST was never intended to give an “independent” measure of anything. It’s always been known BEST uses some of the same data other group use. That means it can’t be independent. That doesn’t make it worse. It just means we can’t call it an “independent” measure of global temperature.
I mostly point that out because people like Michael Mann misuse “independent” like this all the time. Mann takes it so far he calls work done by his co-authors “independent” confirmation of his work. He also likes to call work “independent” confirmation of his even if half of its data was used in his work as well.
I can’t criticize Michael Mann for misusing “independent” like that and not speak up when I see others do it too.
Then you can’t take averages, summaries, or anything like that. All such procedures throw away the vast majority of the “raw data.” There’s no automatic, objective way to decide which bits of the “raw data.” to keep while throwing away the vast majority of the rest.
But that’s the entire point! Throwing away raw data is generally verboten, but if it is necessary, it has to be very well justified using a procedure that is determined in advance. Berkeley Earth has come up with a reasonable algorithm, which I am willing to trust. I just don’t think that the processed data should be treated in the same way raw data would be.
Zeke (Comment #130058) June 7th, 2014 at 11:45 am
“Mosh, Actually, your explanation of adjusting distant past temperatures as a result of using reference stations is not correct. NCDC uses a common anomaly method, not RFM.
The reason why station values in the distant past end up getting adjusted is due to a choice by NCDC to assume that current values are the “true” values. Each month, as new station data come in, NCDC runs their pairwise homogenization algorithm which looks for non-climatic breakpoints by comparing each station to its surrounding stations. When these breakpoints are detected, they are removed. If a small step change is detected in a 100-year station record in the year 2006, for example, removing that step change will move all the values for that station prior to 2006 up or down by the amount of the breakpoint removed. As long as new data leads to new breakpoint detection, the past station temperatures will be raised or lowered by the size of the breakpoint.
An alternative approach would be to assume that the initial temperature reported by a station when it joins the network is “true”, and remove breakpoints relative to the start of the network rather than the end. It would have no effect at all on the trends over the period, of course, but it would lead to less complaining about distant past temperatures changing at the expense of more present temperatures changing.”
Any algorithm that does this will bias the past cooler and cooler and worse as it goes on. Lucky the postcasts do not go back another 200 years or we would be in an Ice age. Also those warm seas back then. R Gates will be happy to know ocean warming travels back in time and heats the seas in the past before 1950 , might explain why we do not have to worry about the future.
No it won’t. ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/williams-etal2012.pdf
Erm, that was supposed to be in a reply to angech2014 above ^
“It’s minor because on average the biases that require adjustments mostly cancel each other out.”
Average, adjustments, mostly.
I see. The raw data is useless as is. By performing mostly random average adjustments, inaccurate and unreliable data is magically transformed into accurate and reliable data.
More or less the same as running a toy computer model several thousand times varying inputs randomly, and then insisting the different useless incorrect – by definition – outputs somehow cancel each other out, and wrong is transformed into right.
The joys of Warmist science! Big is small, minor is large, adjustments balance each other out, but we need them anyway, and so on.
In any case, what’s the point? On average, for four and a half billion years the Earth has cooled. On average, over the last twenty years or so, the Earth has cooled around twenty millionths of a degree or so. Gee!
What a waste of time! I hope you do all this at your own expense, or accept only voluntary donations from supporters.
Live well and prosper,
Mike Flynn.
If you want raw data they by all means USE IT.
you will actually WARM the record after 1960.
I have been trying to follow this argument for a long time. What needs to be addressed are the comments that adjustments are “inconsequential” but would result in consequential warming if removed. Makes no sense to me. I guess some folks still consider warming of 0.02C is consequential. Being a student of climate history, I still think the viking settlement of Greenland, the well-documented temperature changes in Iceland, fishing records that show movement of species and many other records show that nothing unusual is happening. The northwest passage must have been open when it was discovered! At times I think we have replaced common sense with a boatload of formulas.
So what? Warm it a bit, cool it a bit, that’s what actually happened in the 1960s and 1970s until we got into the 1980s and it started to get warmer still.
I’m with Mike, history doesn’t cancel out. BEST and the rest can adjust and homogenise all they like.
R2, you will like essay Northwest Passage. Mostly ‘selfies.!
Mike Flynn: By performing mostly random average adjustments, inaccurate and unreliable data is magically transformed into accurate and reliable data.
More or less the same as running a toy computer model several thousand times varying inputs randomly, and then insisting the different useless incorrect – by definition – outputs somehow cancel each other out, and wrong is transformed into right.
They have explained before what they did. And they’ll explain again, according to their post. That was not what they did.
Which files are you getting from NIWA, because if it is their official files then they “adjust” the temperatures before you get them. They do have the REAL raw data as well, so you may have the right files, but I just wanted to check.
As with most of the world, it would appear that NIWA’s adjustments, which should in theory be random, seem to be a significant “cause” of the global warming experienced here – ie the same story as Brookers. It would seem to me that, since that was the case, NIWA scientists would be falling over themselves to have the quality of their adjustments peer reviewed, but in fact they fought a court case to prevent anyone getting access, except of course for the Australian BoM which has (going by the Darwin example anyway) the same issue of having most of the global warming coming from “neutral” adjustments.
I am sure you are familiar with both situations.
Mary
daily unadjusted data.
here is clue. take NZ. make all the temps 0
the global average wont change.
Good grief Mosher, talk about Exhibit A evidence for climate science’s utter contempt for locality and history.
I really am surprised by that, given the records for Campbell Island are amongst the longest and best for the sub-Antarctic and for Raoul Island amongst longest and best for the sub-tropical, and the Chatham Islands amongst the longest and best for the sea to the south east of New Zealand. “New Zealand” is of course not just the landmass, but the wider region.
However, I have never looked at it. I am pleased that you are using the old unadjusted numbers. If you ever want to verify they really are unadjusted, they were published each year, from 1893 onwards, in the official year book, which is online here: http://www.stats.govt.nz/browse_for_stats/snapshots-of-nz/digital-yearbook-collection.aspx
“the global average wont change.”
for one country? no, not significantly.
The point is that BEST, like most attempts at what they did, assume that the “raw” data is actually raw. For NZ and Australia, it is well documented that it is not, and well documented that the adjustments changed the trend – even changed the SIGN of the trend. Nor was this change “insignificant” – in at least one case >2C change in reported trend!
So, given that we have 2 examples of this “corruption” of “raw” data, at what point is it valid to ask: how many countries do this and how does this affect any attempt at analysis (inc BEST).
How many countries do I need to demonstrate do NOT supply “raw” data for their “raw” records before it is an issue you feel is worth persuing?
Just curious…
Mary
>”Which files are you getting from NIWA, because if it is their official files then they “adjust” the temperatures before you get them. They do have the REAL raw data as well, so you may have the right files, but I just wanted to check. ”
Mosher
>”daily unadjusted data.”
Yes, raw unadjusted data from NIWA’s CliFlo database.
Now the fun starts. All the institutions, NIWA, GISS, BEST, (NiWA provide CRU with adjusted 7SS data – see below), select different stations, same stations different timeframes, more or less stations, different homogenization methodologies, etc, from the CliFlo database (and thence distributed elsewhere with the added complication of GHCN) either directly or indirectly.
The NIWA product that corresponds to BEST’s kriging methodology is their multiple station Virtual Climate Network (VCN). VCN predates BEST i.e. NIWA were implementing a method similar to BEST’s before BEST got around to it, and selling it, it’s proprietary. VCN only starts at 1972 because CliFlo records from then on are good quality in NZ and the VCN series profile corresponds to BEST 1980 – 2000 except the respective effective altitudes are a little different therefore an absolute offset.
The VSN and BEST trends since 1972 are essentially 2 cooling periods interrupted by a 0.45 C spike in 1997 giving an overall warming trend (which BEST describes by +linear trend – don’t mention the cooling). There is no locality homogenization of stations in the VCN, each station is a separate entity. NIWA’s CO2-forced model-based forecast from 1990 (see their website) is already tracking too warm relative to VCN and BEST (and the 7SS).
Except the VCN is not what NIWA provides CRU for CRUTEM4/HadCRUT4, that’s their 7SS which homogenizes multiple stations to compile a locality series e.g. Auckland. Albert Park is the long-running Auckland series but is UHI/Sheltering contaminated so the series moves to Mangere Treatment Plant and then to Mangere Airport. NIWA didn’t correct Albert Park for UHI/Sheltering, NZCSC did correct. NZCSC arrive at a 7SS series with a trend that is one third of NIWA’s.
BEST continued to use Albert Park for Auckland but not Mangere (why not?). But even then, after adjustments and homogenization, BEST’s Auckland trend (Albert Park) is closer to the NZCSC Auckland trend (Albert Park + Mangere) than to NIWA’s. If BEST corrected Albert Park for UHI/Sheltering, their Auckland trend would be lower than NZCSC’s and far less than NIWA’s i.e. corroborating NZCSC but not NIWA.
BEST pull in Albert Park as the primary data for Hamilton which is 125 km south in a different climate zone when they could use Ruakura Research situated in Hamilton (why not?) that NIWA uses for their Eleven Station Series (11SS).
GISS appear to be thoroughly confused with Auckland, their GHCN datasets prior to final are here:
http://data.giss.nasa.gov/cgi-bin/gistemp/find_station.cgi?dt=1&ds=13&name=Auckland
Auckland, Auckland Airport, and Auckland Albert Park. I don’t know what “Auckland” is, it’s only a few years. Albert Park should be long-running but they only pick up 6 years. The airport site was only used as a club airfield from 1928, airport built 1966, NIWA pick up the airport site in 1998. After Albert Park, NIWA picked up Mangere Treatment Plant from 1976 – 1998. GISS seem to think the airport has been operating continuously from 1879 to 1990 (see link above). Odd, because the Wright brothers first flight was 1903
The GISS final Auckland dataset is here (Airport):
http://data.giss.nasa.gov/cgi-bin/gistemp/show_station.cgi?id=507931190000&dt=1&ds=14
Except the final series (Airport) runs from 1950 to 1991 instead of 1879 above. But at least the airport was actually operating from 1966. I doubt that “Airport” data is from the airport.
So you see Mary, even if the same raw data source is accessed by the respective groups in this New Zealand example, the final results from each are widely dissimilar.
Ah yes, the old “we adjusted the data because” excuse…
In the engineering field measurements that are made for mission critical applications are traceable to a national standard issued by a standards laboratory. I have participated in work like this, it often involves purchasing a “transfer standard” instrument. For example a “standard” yardstick. Then through a careful process of tracking and accumulating errors through each step in the process it is possible to state that; “these measurements are traceable to the US NIST standard (for length as an example) within an error (1 sigma) of plus or minus 1 micron (for example).
None of these calibration and traceability steps have ever been applied to the “official” temperature record. It is a “dog’s breakfast” of garbled data with adjustments added in ad-hoc to make it match the “expected” results.
The biggest example of confirmation bias is the observation that almost all of the adjustments make the distant past colder and the recent past warmer…. The chances of that happening in a system with randomly distributed errors is huger than your chances of winning the lottery. Of course it appears that some climate scientists have won the tax payer funded lottery; Hey give me some grant money and I can give you something to worry about, and I can make it look like the END OF THE WORLD AS WE KNOW it, oh and we need to study it more to make extra sure the world is coming to an end….
The surface temperature record is worthless at this point in history, and no amount of tax payer dollars can ever turn it into a useful data set to base policy decisions on.
If you are flying on an airplane are you reassured that all the companies that manufactured all the myriad components (engines, wings, fuel tanks, etc…) did their work with calibrated measuring instruments that are traceable to a single national standard (NIST in the US), or are you OK if they just “adjusted” their measurements so their portion of a complex machine “looks OK with our measurements” ???
Cheers, KevinK
Kevin K: Re: Chances of “almost all of the adjustments make the distant past colder and the recent past warmer” is very small.
I agree with your sentiment here, but have a possibly different explanation for the cause. I disagree with Mosher on many things, but I trust his honesty on temperature records ( recognizing that he can make mistakes just like anyone else). I am beginning to think that the Hansenite researchers are so incompetent and closed minded that they unknowingly clung to substandard methodologies which unwittingly undermined their goal of cooling the past and warming the present.
JD
JD, “I am beginning to think that the Hansenite researchers are so incompetent and closed minded that they unknowingly clung to substandard methodologies which unwittingly undermined their goal of cooling the past and warming the present.”
Note really. Their method, long range interpolation produces a different product. In some case it is a “better” product, some cases not so much.
It is pretty much a waste of time arguing about the “adjustments” unless you have a specific example of when interpolation is good and when it isn’t so good. For the climate of say Kansas, Interpolation not so good because Kansas temperatures would be “adjusted” to New Orleans climate. A Kansasite ( whatever) might not care for that.
Here is a Redneck take on the subject,
http://redneckphysics.blogspot.com/2015/02/whats-deal-about-global-temperatures.html
==> “On balance the effect of adjustments is inconsequential.”
I refuse to believe that. If that were true, then I couldn’t argue that the adjustments were made in order to scare the public
But I know that the adjustments have been made in order to scare the public. Therefore, it is obvious that the effect of the adjustments have been to create warming effect.
It doesn’t matter what the data actually show. When will you get that through your head?
Its pretty funny.
The say they believe in climate change. ( just not human caused) I show them data that CONFIRMS their belief that claimate changes… and they reject data that confirms their belief… weird
Yeah, in figure 1 the adjustments clearly reduce the average(s) prior to 1900, which suggests an increased magnitude to the warming coming out of the LIA. Assuming you can tell anything about temperature trends that far back, given the sparsity of the measurements.
What stands out to me is how figures 5-8 show that the adjustments pull most of the warming into the high NH latitudes, where it’s more likely to make things more comfortable than less. For instance, all four figures show significant (to the eyeball) warming in southwestern Africa without adjustments, which almost goes away once the adjustments are made.
Once people start actually looking at what the adjustments do to the impact, I’d expect it to be the alarmists who start yelling about a “conspiracy” to “hide the impact” of greenhouse changes. Come to think of it, didn’t BEST get some help from the Kochs?
Steven Mosher, “Its pretty funny”
It actually is, that is why I enjoy watching the silly back and forth. weren’t you at one time going to see how much interpolation of coastal stations impacted Berkeley Earth’s final product? If a young paleo kinda guy is trying to calibrate his reconstruction of temperature is the wilds of BFEgypt, would he really want Atlantic City temperatures influencing his results?
“It actually is, that is why I enjoy watching the silly back and forth. weren’t you at one time going to see how much interpolation of coastal stations impacted Berkeley Earth’s final product?
We looked at that in a couple of different ways. Distance from Ocean
and then distance from large body of water. Because you know
some guy in western michigan would argue ” but Holland and Grand Haven!!” Distance from coast worked to improve estimates of seasonility.
Basically.. the temperature isnt different.. the RANGE of variability is.
Hmm I did some work with that long ago.. looking at how variance
changes with distance from coast.
We may have more luck with cold air drainage as I have found a dataset
that saves me about a YEAR in compute time.
Blogging bozos have no idea the huge amount of time is involved in
driving an estimate from 1 degree resolution down to 1km. its staggering.
If a young paleo kinda guy is trying to calibrate his reconstruction of temperature is the wilds of BFEgypt, would he really want Atlantic City temperatures influencing his results?
Depends how good his local temps are.
It’s all a matter of attribution. Wierd that that would matter.
===========
“Its pretty funny.
The say they believe in climate change. ( just not human caused) I show them data that CONFIRMS their belief that claimate changes… and they reject data that confirms their belief… weird”
BEST loses when Mosher strikes out. He pollutes the atmosphere and it’s not only with CO2.
I compared one Swedish station with data from SMHI (Swedish Met office) with the data from BEST, I didn’t understand anything. SMHI had one serie, BEST three, when I tried to look at specific years, there was few similarities. How is the quality check done? Where have BEST get their data from? What have happen to the data from SMHI before it reached BEST?
HenrikM,
Since no one responded I will put one idea out there. I have read Mosher’s description of the BEST procedure and one thing I believe they do is to split a temperature record if there is a station location change and give it a new number. This was intentional to do it a different way than other groups had (rather than just adjusting the data?). So, the 3 you mention may be just this procedure showing up.
the data comes from 14 different sources, hourly, daily, monthly.
Those 14 different sources can contain multiple records for the same station.
Step one of our procedure is to try to eliminate duplicate records from different .
records can be combine as duplicates if
A) the location is a close match AND
B) the name is a close match AND
C) the data reported is the same.
If stations cannot be combined, they are kept as separate stations.
In some case ( Norway, sweden, etc ) there are cases where a human
might look at the electronic records and other non public records and
conclude that there is only one station.
As local data providers clean up their metadata ( for example WMO asked people to re survey sites ) there is room for the station deduplication process to be improved.
There are two kinds of error in station de duplication.
1. Failing to merge two stations which are “really” the same record.
2. merging two records that are really different.
We do everything to AVOID the second error because merging two records that are different will just cause the breakpoint algorithm to break them apart. And errors of type 1, dont change the values computed, they do
make you think you have more stations that you really do.
Hmm, a recent improvement to the algoithm eliminated about 400 of these errors. there wasnt any effect on the answers.
for folks who work in database land.. you can think of the problems and techniques for doing a probablistic joins.
Again, an interested commenter notices that BEST’s amazing global temp diagram doesn’t actually match anything that actually happened at a locality, doesn’t reflect that locality’s history at all. Mosher’s response, blah blah error matching, something something. Whatever.
By the way, it’s 16 data series at BEST website, Mosh. Not 14. Not good with numbers, eh.
Hide the decline
I have made the point many times that the world comprises many different climates and trying to jam them all into one global average means we miss all the nuances evident in the local records.
Local is after all where we all live and that is where we need to know what the trend is and why.
Tonyb
“Since the algorithm works to correct both positive and negative distortions it is possible to hunt through station data and find examples of adjustments that warm. It’s also possible to find stations that are cooled.”
Please provide the numbers cooled vs. warmed.
“Please provide the numbers cooled vs. warmed.”
You can find multiple GHCN histograms here. Or find the stations themselves on this Google Maps app.
lance the NUMBER will fool you.
look at the black line.
of course we could do no adjustments. Then we would have a HIGHER trend from 1960 to today.
==> “of course we could do no adjustments. Then we would have a HIGHER trend from 1960 to today.”
Maybe so – but still, you’re only adjusting the data in order to scare the public about warming.
See, no matter what you say I can still make that argument.
Sure you can make stupid argument, Joshua. Nobody ever doubted that.
Mosher said” lance the NUMBER will fool you.
look at the black line.
of course we could do no adjustments. Then we would have a HIGHER trend from 1960 to today.”
The thing is, it’s double-edged sword. You make it steeper and the pause is more profound.
You can’t cheat your way out of everything.
“And next a chart for those who want to accuse the algorithm of cooling the planet”
Here is my corresponding plot for continents. It also shows adjustments mostly warming Africa. There is an active version here. Using TempLS mesh – ie GHCN adj/unadj and ERSST.
http://www.moyhu.org.s3.amazonaws.com/trend/breakdown/break3.png
Am I missing something or does this chart show that GHCN cools the past?
Yes. But with that quote, the OP was saying that it isn’t really cooling Africa. And it isn’t. The plot shows the general cooling by up to 9.95°’C.
The most warming places are where tthere are fewer stations.
BEST likes it that way.
Figure 2 on the US shows that the most recent temperatures (after 1980) require the most adjustment upwards. Shouldn’t we be getting more accurate rather than less with our temperature measurements?
Hi Lance,
In the 1980s the U.S. switched the vast majority of the sensor network from liquid in glass thermometers to MMTS thermometers, which resulted in a sizable cooling bias: http://rankexploits.com/musings/2010/a-cooling-bias-due-to-mmts/
Similarly, time of observation was changing for many stations over the same period: judithcurry.com/2014/07/07/understanding-adjustments-to-temperature-data/
In fact, the U.S. post-1980s was unusually inhomogeneous in many ways; most of the rest of the world was much less so.
Zeke
Thank you for your efforts and excellent work. I know we give Mosher a lot of grief but I think most appreciate his very hard work to produce a quality product.
I come at this not from the statistical perspective, but from a large organizational view. Many, many thousands of individuals have been involved over the course of hundreds of years to observe, enter and archive data. ( I see one of your locations in Germany had temperatures into the 1700s) While everyone involved at those 40,000 locations over the last few hundreds of years may have been doing their best, I just wonder how many entries were somehow not reflecting reality of what the actual temperature was that day. In spite of the great intentions of every person recording the temperatures and trying to get it with accuracy, there are going to be errors. Even if it is just writing down one number when thinking of another number, which I do all the time with Sudoku. This concern goes beyond the Time of Observation to simply the probability of not getting things right.
This is not something that anyone can solve, so it certainly is not your problem or anyone else’s problem. But it is one of the areas that call into question how close our records are versus what actually happened in the field.
Thanks again for your continued good work.
ceresco.
That is a real problem. There are several approaches.
First one has to make the stark admission that we will never have certainty about the past. All we have are records. It doesnt matter whether the record was signed, or checked, or claibrated, or whatever. it is a record. it is not the temperature. it is a record of the temperature. Further there is no way to go back and time and check it.
Approach number 1 is the phil jones approach. he does a bottoms up
estimate of the errors. For transcription error he has ( as I recall ) a really small number.
Approach number two TOP DOWN.
we calculate the error in a top down fashion and say the sum of all errors is X. We use the correlation function to do this ( look at the nugget )
This error contains all error whatever the cause. Its much larger than a bottoms up error. One of our british reviewers didnt like this large error.
Fig 4 gives a better idea. And if you compare the two, CONUS is mostly metadata based – I suspect TOBS. It corresponds to the rate at which TOB was changing.
Bias corrections aren’t (mostly) correcting inaccurate measurement. They fix inconsistency. You can measure accurately at 5pm and 9am, but they aren’t consistent. Neither is wrong.
When I was at high school, the weather station I monitored for a term used thermometers that pushed a marker up to save the previous days max temp, & another thermometer which pulled a marker down to save the overnight minimum. Unless the US Weather Service chose not to use this type of thermometer in their network, it beats me how TOBS could make any difference to the daily readings. How can recording the daily maximum & overnight minimum at 9:00am, or 5:00pm make any difference to the actual max & min? If you ask me the adjustments for TOBS are dubious. This punter is genuinely perplexed.
There is an explanation here.
Robert, Zeke, Steve:
Thanks for this.
The question is not whether homogenization is necessary. It is.
The question is whether homogenization is done correctly, a question that is hard to answer, and I would worry about the impact it has on the correlation structure.
A more important question is perhaps why the public lost so much faith in climate science that they prefer to believe Booker over you guys. The Telegraph poll suggests that 90% of 110,000 readers are with Booker.
I would hypothesize that the constant stream of climate nonsense — it’s five for twelve, kids will not know what snow is, we’re all gonna die, last chance to save the planet, climate change is coming to blow over your house and eat your dog — has made people rather suspicious of anything climate “scientists” say.
If my hypothesis is correct, instead of arguing with Booker about the details of homogenization, you should call out the alarmists.
Hi Richard,
When Shakhova argues for a “methane bomb” in the arctic, I loudly disagree.
When Wadhams argues for the arctic being ice free in the next few years, I loudly disagree.
When Booker says that station adjustments are the “biggest scientific scandal ever”, I loudly disagree.
Maybe its selection bias on my part, but I more often come across the Bookers of the world misrepresenting my area of focus (surface temperature records) than those who are on the “alarmist” side of the fence. I’m sure if my area of expertise was arctic methane the story would be different :-p
Shakhova argues for 50 Gt of methane that is eligible for early release.
Currently 0.5 Mt of methane is released by hydrates. Unless the release rate increases to over 1000 times the current rate it isn’t even an issue.
About 650 Mt of methane is released per year with about 55% related to humans in some way (mostly rice and livestock).
1820 PPM (5.3 Gt) more or less is the current atmospheric methane level with about a 10 year lifetime.
The pre-cambrian methane level was 1000 times higher than today.
The cold reality is unless all 50 GT of unstable methane is released over a few years time, it won’t make any practical difference and if there is a rapid release, the atmospheric levels of methane will rapidly decline afterward..
.
You are right, Richard. It’s climate scare fatigue. I have the habit of looking up into the sky and saying to people I encounter here, there and everywhere “Global warming.” Pouring rain, snow, hot, cold, whatever “Global warming.” Almost always they smile and roll their eyes.
“Could be wrong but I thought I saw Steven Mosher try to marginalise the importance of GCM’s above in this:”
No.
I am saying this.
Take GCM results: compare them to adjusted data. Draw conclusions
about how poor the models perform
Take GCM results: compare them to raw data. Draw THE SAME conclusions about how poor the models perform
get it.
Take any critical question in AGW.. change the data from raw to adjusted
Your answer wont change in any material fashion
Richard???
Maybe I should write a book on climategate?
Zeke, Steve:
I stand corrected by both of you.
“I would hypothesize that the constant stream of climate nonsense — it’s five for twelve, kids will not know what snow is, we’re all gonna die, last chance to save the planet, climate change is coming to blow over your house and eat your dog — has made people rather suspicious of anything climate “scientists” say.”
Exactly. Thank you.
> The Telegraph poll suggests that 90% of 110,000 readers are with Booker.
In other news, water is wet:
http://abs.sagepub.com/content/early/2012/09/12/0002764212458274
But it’s a self-selecting poll, Richard. First you have to go to the Telegraph site, then read Booker’s column, then you have to give a damn enough to do the poll. Just a look at the comments below the article would show which way it was going to go. But then the Telegraph was home to James Delingpole too and is “rather to the right” (euphemism) on the political spectrum, so it is no surprise that its self-selecting survey would produce a result like that.
Of course. It is still a staggering amount of support for what is a rather stupid proposition.
It’s a completely unscientific sample, and doesn’t justify any kind of conclusion, but nonetheless Richard can use it to test his “hypothesis” about what “the public” believes in contrast to what is shown by empirically collected data w/r/t public opinion – and to find support for…..well….nothing really except confirming his bias.
Nice science, that, from someone who’s concerned about activist scientists.
Except that the question asked was “Do you think Scientists have exaggerated the threat of global warming?”, not “Do you think Scientists have manipulated the temperature data to exaggerate warming.”
It’s perfectly reasonable to answer yes for the first question and no for the second, which is likely what has happened. I conceded that in the context of the article it’s not unreasonable conclusion to draw, but I suspect for those who read the article, it wasn’t the first they have read on the issue that informed their views.
Richard Tol (@ Richard Tol): I would hypothesize that the constant stream of climate nonsense — it’s five for twelve, kids will not know what snow is, we’re all gonna die, last chance to save the planet, climate change is coming to blow over your house and eat your dog — has made people rather suspicious of anything climate “scientists” say.
I think you make a sound point about a problem that these authors have not themselves exacerbated in any way. However, the other important points are: (a) nothing in statistics conforms to intuition or common sense, and Booker appeals to common sense in pointing out the differences between the raw data and “adjusted” data in a few select extreme cases (b) few people have even tried to master the details of the various algorithms
(there is no accurate, intuitive, non-technical description of the BEST procedure possible.)
Re: Telegraph poll. Sample bias. Can’t make conclusions based on results of that poll.
Do I really need to spell this out?
Do I think that the Telegraph is a peer-reviewed, learned journal? No.
Do I think that the straw poll was designed and tested to the most rigorous standard? No.
Do I think the result has external validity? No,
That said, after being prompted by a story about a scientific scandal, readers were asked their opinion. 89% agreed with the author. That means that, at that moment in time, to that readership, Booker told a convincing story.
Ask the same crowd 15 minutes later and you might well get a different answer. Ask a different crowd and you might well get a different answer.
But that does not take away from Booker’s power to convince a large number of people to at least temporarily believe a story that is just not true.
Do I think that Booker can convince 89% of his readers that the Queen really is a lizard from outer space? No.
Good point, RIchard Tol – but Mosher is all about attacking the skeptic he makes up for the occasion. He needs that skeptic.
The only place he doesn’t cheat outrageously in argument is in front of Steve Mc.
If the adjustments are really so “minor” as to be insignificant, why make those adjustments at all?
That’s probably a don’t ask, they won’t tell question. But I know that for some it’s fun, for others it’s profitable.
Don
Nearly four years ago I wrote this article on the general unreliability of historic temperature records and how we should not believe they can be accurate to fractions of a degree. This 110 year old book is still worth reading as it highlights the problems inherent in constructing any temperature record, let alone a historic one.
“The skill and diligence of the observer were of course paramount, as was the quality of the instrumentation and that a consistent methodology was employed, but this did not prevent the numerous variables conspiring to make the end result-an accurate daily temperature reading-very difficult to obtain. Indeed the errors inherent in taking measurements are often greater than the amounts being measured.
Many of these basic concerns can be seen in this contemporary description from a 1903 book which relates how temperature recordings of the time were handled. The “Handbook of Climatology” by Dr Julius von Hann (b. 23 March 1839 d. 1 October 1921) is the sometimes acerbic observations of this Austrian, considered the ‘Father of Meteorology.’
The book touches on many fascinating aspects of the science of climatology at the time, although here we will restrict ourselves to observations on land temperatures. (It can be read in a number of formats shown on the left of the page on the link below).
http://www.archive.org/details/pt1hanhdbookofcli00hannuoft
This material is taken from Chapter 6 which describes how mean daily temperatures are taken;
“If the mean is derived from frequent observations made during the daytime only, as is still often the case, the resulting mean is too high…a station whose mean is obtained in this way seems much warmer with reference to other stations than it really is and erroneous conclusions are therefore drawn on its climate, thus (for example) the mean annual temperature of Rome was given as 16.4c by a seemingly trustworthy Italian authority, while it is really 15.5c.”
That readings should be routinely taken in this manner as late as the 1900’s, even in major European centres, is somewhat surprising.
There are numerous veiled criticisms in this vein;
“…the means derived from the daily extremes (max and min readings) also give values which are somewhat too high, the difference being about 0.4c in the majority of climates throughout the year.”
Other complaints made by Doctor von Hann include this comment, concerning the manner in which temperatures are observed;
“…the combination of (readings at) 8am, 2pm, and 8pm, which has unfortunately become quite generally adopted, is not satisfactory because the mean of 8+2+ 8 divided by 3 is much too high in summer.”
And; “…observation hours which do not vary are always much to be preferred.”
That the British- and presumably those countries influenced by them- had habits of which he did not approve, demonstrate the inconsistency of methodology between countries, cultures and amateurs/professionals.”
http://wattsupwiththat.com/2011/05/23/little-ice-age-thermometers-%E2%80%93-history-and-reliability-2/
tonyb
I’ll take a look in the morning, tony. I am with you on the hundredths of a degree thing. Not feeling that.
vuurklip,
I guess if you are described as an “. . . energy systems analyst and environmental economist with a strong interest in conservation and efficiency.”, it might make you look sciency. Got to keep those donations rolling in!
Nobody provides funding for totally useless data adjustments, do they?
Live well and prosper,
Mike Flynn.
Seen my comments over at ATTP ( ill copy them here)
But before I do that.
1. historically we were asked or challenged to investigate the charges that
people had their thumbs on the scale while doing adjustments.
2. when you start you dont know what you will find
3. you report what you find
Now from ATTP
“I suppose one does the adjustments to eliminate the possibility that there is some bias in your answer that would cascade into questions that really matter. So what do I mean by really matter.
1. There is nothing in the adjustments that will change the record in such a way as to influence our confidence in AGW. That is, with or without adjustments our confidence remains the same.
2. There is nothing in the adjustments ( use them or not) that will cascade into climate reconstructions in such a way as to change our perceptions of say the LIA or the MWP in any
significant way.
3. There is nothing in the adjustments (use them or not) that will modify in any material way the consensus position on attribution.
4. Nothing important about sensitivity that will change. For example, doing or not doing them will
not change the envelop of uncertainty in any material way.
5. Nothing in the adjustments ( use them or not ) will materially effect ones view of how well GCMs
hindcast or forecast.
So one might, for example, use an unadjusted series and adjusted series in say ‘Curry and Lewis”
My take is this. That paper wasnt a paradigm shifter before, and it would not be a paradigm shifter after. Numbers might squish around a bit, but you are not going to find out that Lindzen was right on sensitivity. because he was not .
so you might paraphrase what I mean by materiality as the following: some important core position . To put it starkly, C02 will warm the planet regardless of what I do to the temperature series. we knew it would warm the planet before anyone compiled a series and we know regardless of what anyone does to it.
I suppose one could argue that the temperature series has some temporary leverage into core beliefs because of the muting of trends in recent years.”
I asked why adjustments are necessary considering that they are so insignificant. I cannot discern any justification for adjustments in your above response.
@ Steven Mosher
“There is nothing in the adjustments that will………..” points 1 through 5.
There IS something in the adjustments that CONSISTENTLY magnifies the ‘problem’.
“To put it starkly, C02 will warm the planet regardless of what I do to the temperature series. we knew it would warm the planet before anyone compiled a series and we know regardless of what anyone does to it.”
To put it starkly, when I cooked hamburgers on my gas grill last summer, I DID warm the planet, no matter what anyone does to the temperature series. I knew it would warm the planet before anyone compiled a series and I know it regardless of what anyone does to it.
In the absence of a firm understanding of the MAGNITUDE of the warming, showing warming as a function of ACO2 (since that is the only part of total CO2 that we can potentially control), what does EITHER statement have to do with the proposition that ACO2 poses an existential threat to the biosphere that demands the establishment of a world wide ‘Climate Change Policy’, giving governments the power to control and/or tax every activity with a ‘carbon signature’, with the magnitude of the carbon signature to be controlled and/or taxed determined solely by the governments doing the controlling and/or taxing?
One of the posters here has written a paper (http://agwunveiled.blogspot.com) which claims that the historical temperature anomaly can be reproduced with a correlation coefficient of .95 by considering ONLY the time integral of the sunspot number. CO2 is ignored completely. I have no idea if he is correct or not, but does anyone even CLAIM that they can reproduce the historical temperature anomaly BETTER as a function of CO2? I would be surprised if they could, since CO2 has been increasing monotonically since we began measuring it, while the observed planetary temperature (however defined, if it ever has been) hasn’t been doing ANYTHING monotonically over the same time frame.
Bob
‘There IS something in the adjustments that CONSISTENTLY magnifies the ‘problem’.”
SST adjustments COOL the record.
the ocean is 70% of the planet.
So lets use raw ocean.
The problem will get magnified like you cant imagine.
Ya,, team raw data.. that’s a hat-trick owngoal.
Next the adjustments do NOT consistently magnify the problem.
we cool the land from 1960 on.
Go ahead.. ask me what we did to the precious pause.
did we magnify the pause?
Steven Mosher, “SST adjustments COOL the record.”
You have said that a dozen times, how much?
“If the adjustments are really so “minor” as to be insignificant, why make those adjustments at all?”
my guess
subconscious attempt to strengthen the argument for a strongly held belief
Dont want adjustments?
Fine. The adjustments in SST cool the record.
Change that and maybe GCMs will look better.
Change that and Curry and Lewis becomes trash.
John Smith (it’s my real name): my guess
That’s a guess.
Tell me what I am missing, Steven:
https://twitter.com/hausfath/status/564939671247941632
It looks like the SST adjustments have warmed the raw temps, where it is evident.
I enlarged it some more and I can see a little downward adjustment in the baseline period, but still?
John, a lot of work on temperature records is in response to previous criticism.
vuurklip: If the adjustments are really so “minor” as to be insignificant, why make those adjustments at all?
Why adjustments were computed has been written about at length, and includes such things as replacing and moving thermometers.
That the adjustments have made such a small difference to the overall trajectory of the mean was not known until after the adjustments were computed.
So, moving termometers to different settings has an insignificant effect? Why move them in that case?
The state of Denmark comes to mind …
vuurklip: Why move them in that case?
That’s only known on a station-by-station basis. The BEST team is stuck with the fact that they had been moved (or replaced), and have to work with that.
They’d miss station moves which don’t produce a break in the record, and there no reason to think the new location would record the same series as the old one was recording
==============
The fact that they need to cheat to scare us really does not help them much.
‘Moist enthalpy hereafter referred to as equivalent temperature (TE), expresses the atmospheric heat content by combining into a single variable air temperature (T) and atmospheric moisture. As a result, TE,
rather than T alone, is an alternative metric for assessing atmospheric warming, which depicts heat content. Over the mid-latitudes, TE and T generally present similar magnitudes during winter and early spring, in contrast with large differences observed during the growing season in conjunction with increases in summer humidity. TE has generally increased during the recent decades, especially during summer months. Large trend differences between T and TE occur at the surface and lower troposphere,
decrease with altitude and then fade in the upper troposphere. TE is linked to the large scale climate variability and helps better understand the general circulation of the atmosphere and the differences between surface and upper air thermal discrepancies. Moreover, when compared to T alone, TE is larger in areas with higher physical evaporation and transpiration rates and is more correlated to biomass seasonal variability.’
https://pielkeclimatesci.files.wordpress.com/2011/11/nt-77.pdf
Homogenize that. It’s the reason surface temperature datasets are obsolete for climate purposes.
Here’s GISTemp and UAH – http://www.woodfortrees.org/plot/gistemp/from:1984/plot/gistemp/from:1984/trend/plot/uah/from:1984/plot/uah/from:1984/trend – the trends are pretty similar. Burt for atmospheric energy content – the satellites are a far more inclusive summation. As the differences between T and Te fade with altitude.
Does anyone have global data for moist enthalpy/TE up to the present date?
Pingback: Climategate II? | Scottish Sceptic
I wanted to do another type of checking with the Norwegian raw data. I wanted to make calculations of temperature change within each segment and then somehow weigh the temperature changes together.
That approach was quickly aborted as there existed no such logs over station data for the weather stations in Norway according to our meteorological institution here.
I am looking forward to the explanation of the adjustments of the surface temperatures!
I am also very interested in the adjustments of the Argo data and the sea level satellite data
All these adjustments seems to change data series in the “same way” (you know what I mean). That fact in itself is not proof of any fraud, but is still interesting to look into.
You know, there may still be a bias. If you look for sources of errors that explains a lack of increase, it’s not unlikely that you find just that.
Best regards,
Knut Witberg
Norwegian station data are a mess. I’ve tried getting something sensible for Trondheim, 3rd largest city here. No good, best we can do is our airport 35 km away. Is the difference large? Not really in terms of absolute temps, but in terms of anomalies? Who knows….
Agree. Most stations are either moved, shut down or have incomplete series. So I guess that’s where homogenization comes in. Fine, but I don’t grasp the fact that WMO accepts this. Interpolation of data should be banned. Our new digtial weather instruments have to go through thorough calibration and acceptance tests. And then we just interpolat areas without stations?! In context of discussing hundreths of a degree globally it doesn’t make sense.
The reason that claims such as made by Booker, Delingpole and Homewood gain traction is that they are written in a manner than can be understood by the average punter, who may have some slight back knowledge and perhaps an inherent bias either way.
Reading all this it strikes me that there is no clear rebuttal in a fashion intelligible to those who may have become concerned by the original articles.
How about a version that actually rebuts the original articles and makes the case for the BEST method in a manner that accepts that readers can be intelligent without being climate geeks?
tonyb
Tony, Homewood’s stuff uses the NASA GISS homogenization of GHCN raw. The CONUS reposted above showed the NCDC pregressive homogenization changes to US GHCN by comparing the final results in 1999 to 2011. Steirou’s paper was all GHCN with reasonably complete 100 year records.
I personally checked the BEST result for a number of the more egregious examples used in the essay. In each case, BEST was ‘better’ than GISS or NCDC. For example, Sulina Romania. A Danube delta town of 3000 reachable only by boat. GHCN raw shows slight warming, maybe half a degree since 1880. NCDC homogenization cooled the past so that there almost a 4C trend. BEST gives it a 0.9C trend. Plainly NCDC GHCN adjustment is unjustified.
OTH, BEST Rutherglen research is problematic. This is the Australian station Graham Lloyd wrote about down under. An agricultural research station in continuous operation at the same location simce 1913. Well maintained. Raw shows no trend or slight cooling. BOM Acorn homogenized into 2C warming since 1913. BEST only has data from 1965! And produces 2C warming. Even worse than BOM, and plainly something not right with the input data feed.
Homewoods critique of GISS is two fold, and is the same for both a portion of the Arctic and central South America (Booker’s two printed columns). Both areas are data deficient and were infilled from adjacent regional stations. Those stations generally had their past cooled (and sometimes the present warmed) by GISS homogenization. Those exaggerated trends were then infilled, producing large regional warmings that NASA displays as big red blotches on its end charts, compared to grey (no data) on the ‘raw’ surrounded by no red. And anyone can get all of this and see for themselves, which is why it has been so hard hitting.
If BEST wanted to contribute substance to those specifics, rather than just defend their method as above, they would show their results for these specific regions. They have many more station records than just GHCN, and (based on my spot checks) usually more reasonable automated adjustments. I suspect that were they to do so, they would also show GISS is deeply flawed.
Rud
You know my admiration for Hubert lamb. There is an article about him here
http://wattsupwiththat.com/2015/02/10/transformation-of-the-science-of-climatology-in-like-a-lamb-out-like-a-lion/
We seem keen to erase the past and pile insults on it as being comprised of worthless anecdotes. Climate science has stood still for the last two decades as we have chased models, novel proxies and extraordinarily complex statistical analysis and have neglected looking for the information that can give us pointers as to how unusual the current climate is compared to the past.
From the archives it doesn’t appear to be that strange, indeed it is rather benign. The climate has varied in the past and will vary in the future and why we should be surprised by a temperature rise since the LIA baffles me.
Tonyb
I’m with Rud. BEST uses Rutherglen temps in Oz and only from 1965. It’s nonsense. So now we know BEST contains at least one incorrect data point. Well, since it contains at least one isn’t it possible it contains many more? The answer is yes which is where Delingpole and Booker are coming from.
How about you hold delingpole, homewood, and booker to account for IGNORING the stations where adjustments cool the record?
read this thread here.
No matter how much work we do there will be people who say
1. Oh, the warmist controll the databases.. its fraud down to the bottom.
2. Oh, go check 1.6B records in their paper form.
3. Oh, never mind, look at UHI
4. Oh wait.. .1C changes everything we know!
5. Well, if they dont matter then why do adjustments.
6. Oh wait, average temperature doesnt exist
7. Oh wait, look at UHA
8. What about climategate?
And at the end of the day you want to use the D word, cause the behavior gets very close to the tactics of people who debate the holocaust by focusing on the actual numbers of people gassed. or by asking where the piles of ash are.
You miss the point and become nasty at the same time. In case of Paraguay all stations were substantially warmed by GISS v3.. In the case of the Arctic Region, 19 stations were warmed, four were not, and none were cooled. And those adjusted warm stations were used by GISS to infill much larger regions, in both cases resulting in big red warmings on their global map.. What does BEST show for those regions? That would be a logical way to address Homewood’s very specific regional GiSS critiques.
Getting nasty, D***** mud slinging does not..
Delingpole Booker and Homewood aren’t screeching at me to make me drive less and pay more tax and revere Gaia. So no i don’t have to hold them to account. You always do this Mosh – say it’s up to us to do a better job that you guys have done about x y z issue. It’s not our job out here in commenter land. You just don’t like that many of us have got reasonable doubt about your infallibility.
Rud. I have studied your comment with some diligence. I cannot figure out what D with 5 * after it could mean. 7? sure. But 5?
tonyb: How about a version that actually rebuts the original articles and makes the case for the BEST method in a manner that accepts that readers can be intelligent without being climate geeks?
Consider some attributes of scientific exposition:
clear
accurate,
complete
nontechnical
concise
Which ones do you want to give up? No accurate and concise exposition is likely to be clear to someone who has not mastered the technical language. No accurate exposition is likely to be nontechnical (that’s why the technology and technical language were invented in the first place.)
Understanding statistics requires understanding conditional distributions (p-values, for example, are estimates of conditional probabilities). Conditional probabilities and conditional distributions are non-intuitive, so the first step in understanding statistical methods is to give up on intuition. Booker has an argument that appeals to intuition: it can’t possibly be technically accurate.
Matthew
Journalists and ourselves are perfectly capable of Extracting information from technical reports if they are written in a clear fashion.
Having read literally hundreds of science papers from the mid 1800’s onwards, I would say that generally the early ones are a difficult read.
I have a weighty tome entitled ‘the history of science’ dating from 1886 which verges on pretentious.
. But the ones exploring pure science and without any agenda or objective from the middle period, roughly the 1920’s to the late 1970’s are, on the whole! models of clarity no matter their complexity.
As regarDs many, but not all, papers from that time to the present, i would say their narrative and clarity has declined . Often it is difficult to understand what their conclusions are.
Whether that is because most are for sale and feel they need to fit a certain format, Whether computers have become a substitute for writing, or whether the standard of scientific English has declined I don’t know.
Tonyb
+1 Tony
>”As we will see the “biggest fraud” of all time and this “criminal action” amounts to nothing.”
Could not be more wrong. See Paul Homewood’s post on New Zealand, in particular the GISS vs BEST comparison of Gisborne Aero adjustments starting here:
https://notalotofpeopleknowthat.wordpress.com/2015/02/09/cooling-the-past-in-new-zealand/#comment-37569
BEST make no adjustments over the period of GISS adjustments, 1963 – 2002:
GISBORNE AERODROME AWS
Breakpoint Adjusted Annual Average Comparison
http://berkeleyearth.lbl.gov/stations/157058
GISS make 7 adjustments over that period:
At 1963 the cumulative adjustment is 0.7
At 1968 the cumulative adjustment is 0.6
At 1972 the cumulative adjustment is 0.5
At 1975 the cumulative adjustment is 0.4
At 1980 the cumulative adjustment is 0.3
At 1982 the cumulative adjustment is 0.2
At 1986 the cumulative adjustment is 0.1
At 2001 the cumulative adjustment is 0.1
At 2002 the cumulative adjustment is 0.0
For example, in GISS monthly adj series (see graph below for raw monthly),
The GISS Gisborne Aero 1973 cumulative adjustment is 0.5
1973 monthly raw (top) vs adjusted (bottom)
19.4 18.5 16.2 14.6 12.7 10.0 8.6 10.5 12.3 14.2 17.2 17.2
18.9 18.0 15.7 14.1 12.2 9.5 8.1 10.0 11.8 13.7 16.7 16.8
0.5 difference for every month
The 1974 – 1977 range of common cumulative adjustment is 0.4
1974 monthly raw (top) vs adjusted (bottom)
17.7 20.6 15.1 14.8 11.2 10.1 10.1 8.9 12.1 13.6 15.5 17.8
17.3 20.2 14.7 14.4 10.8 9.7 9.7 8.5 11.7 13.2 15.1 17.4
0.4 difference for every month
1977 monthly raw (top) vs adjusted (bottom)
18.4 18.9 17.8 14.5 10.9 10.1 9.4 10.4 10.2 13.4 14.9 17.5
18.0 18.5 17.4 14.1 10.5 9.7 9.0 10.0 9.8 13.0 14.5 17.2
0.4 difference for every month
The 1978 cumulative adjustment is 0.3
1978 monthly raw (top) vs adjusted (bottom)
19.2 19.5 17.6 16.4 12.0 10.0 9.7 10.3 11.3 12.0 16.0 18.0
18.9 19.2 17.3 16.1 11.7 9.7 9.4 10.0 11.0 11.7 15.7 17.7
0.3 difference for every month
Apparently, according to GISS (but not BEST), there were 2 distinct 0.1 steps from 1978 to 1977 and from 1974 to 1973. Similarly for the other ranges of common cumulative adjustments.
There is no justification for these 2 steps (or the others) in view of the raw monthly data series (and no site moves): http://climate.unur.com/ghcn-v2/507/93292-zoomed.png
GISS has some explaining to do.
Robert Rohde, Zeke Hausfather and Steve Mosher write in the post after plotting global surface temperatures:
“As Figure 1 illustrates the effect of adjustments on the global time series are tiny in the period after 1900 and small in the period before 1900.”
“Tiny” is doesn’t tell us anything.
If you would, please plot the annual global “raw” versus final BEST land surface air temperature anomalies along with their linear trends for the period of 1900 to 2014. The BEST data should have a linear trend of about 0.1 deg C/decade while the “raw” data should have a trend of about 0.08 deg C/decade.
Thank you.
Tiny” is doesn’t tell us anything.
I will make you a bet.
1. Take our raw series.
2. Pick any important science paper you like.. say Curry and Lewis
3. Use the raw data instead of adjusted data.
4. See if the conclusions change.
In other words YOU do some science. You demonstrate how the difference between adjusted and raw CHANGES our understanding.
does it change attribution arguments? nope
sensitivity arguments? Nope
reconstruction arguments? nope
gcm arguments? Nope.
in other words, switching to RAW won’t falsify any important work.
Now, switch to raw on oceans.. everything you ever have done will be junk.
Steven Mosher, thanks for the attempt at misdirection, but it’s not going to work. I asked a very simple question and you’ve gone off on a little rant.
Now, are we to assume by your misdirection that you’re confirming that for the period of 1900 to 2014, the BEST land surface temperature data have a linear trend of about 0.1 deg C/decade while the “raw” data have a trend of about 0.08 deg C/decade? Sure looks like it.
With respect to HADSST3 data and the ICOADS SST data, while both exclude satellite data, they support, not contradict, my ENSO-related findings during the satellite era…or aren’t you aware of that?
“4. See if the conclusions change.”
Oh, so in the same way as Mikey’s method was A OK.
Gotcha.
Bob Tisdale: Robert Rohde, Zeke Hausfather and Steve Mosher write in the post after plotting global surface temperatures:
“As Figure 1 illustrates the effect of adjustments on the global time series are tiny in the period after 1900 and small in the period before 1900.”
“Tiny” is doesn’t tell us anything.
True enough. It is the graph that explains what “tiny” means. Thus, with the graph displayed, you may accept or reject the judgment that the effects are “tiny”. Would you call them “non-tiny”; “a tiny bit more than tiny”; “a lot more than tiny”? Whatever, the graph is there for anyone to judge.
Matthew R Marler, I agree that “tiny” is subjective but I disagree that anyone can judge its significance by looking at that time-series graph…which is why I asked them to confirm the warming rates.
“On this metric, Berkeley, NASA GISS and NOAA are all consistent with GCMs but on the low side of the distribution.” I find this state very curious. It seems to be saying that the data (albeit reconstructed) fit the models. I’m generally much happier when the models fit the data. I suppose climate science is different.
‘Berkeley Earth developed a methodology for automating the adjustment process ‘ this process self aware so has no human input or it is man made in the first place . The fact its automatic after you start it makes no difference to the fact that before it started you make choices has to what it should do . And it is in those choices that the problem lies , for the ‘right’ initial choice can offer you the best chance of the getting the ‘right results ‘
Its part of the mythology of ‘models’ that gives them this notion of being outside of human control , when in fact now matter how powerful the machine running them they can do nothing but what they are told to and see no data they are not given. They actual have less ability to think for themselves than a new born child .
What you may have missed is that we validated the algorithm in a double blind test.
So yes, we made choices.
We tested those choices in a systematic manner.
The choices we made dont change the answer
BUT, if you like you have these options.
1. Use raw data ( the trend since 1960 will GO UP, 2014 will go UP)
2. Create your own algorithm and test it.
Mr. Mosher, you have stated several times above that the adjustments are “small” or “tiny” and don’t have an impact but as you have stated again the trend will “GO UP” if the adjustments aren’t used. Well if your first statement about them being “small” and/or “tiny” is true then wouldn’t that make the “GO UP” tiny? it seems contradictory to argue they are small but then if they are not used it is an “own goal” as they will make the trend “GO UP”
No think harder.
“Think harder”
You’re mentioned in other posts that adding new data, increases warming, do you notice the inconsistency between your double blind tests that don’t change the answer, and new data that does?
Steven Mosher says, Feb.10 at 1:42 am: “GLOBALLY….the adjustments are mousenuts”
Steven Mosher says, Feb.10 at 4:12 am: “you will actually WARM the record after 1960” – about using non-adjusted data.
I don’t really get the hang of those two statements. If the adjustments really are mousenuts, then you don’t warm the record after 1960 whatever you do. If the adjustments are larger than mousenuts (rats, rabbits…?) then you might warm that record. I’m sure if you keep the record with your nuts, you’ll warm the entire period….
Anyways, you have now demonstrated that making adjustments to the record does not contribute anything. So please publish this and make sure everybody stops adjusting the data. You have demonstrated, according to yourselves, that the raw data is more than good enough.
On a final note, your figure 1 and 4 shows that the adjustments, globally, around 1860 are on the order of 0.4 degrees cooling. That is out of a total change of 0.8 degrees. How you can say that this is minor is beyond me. The cooling pre-1900 looks to be between 0.4-0.1 degrees, which is 10-50% of the entire change. It might be that you find this particular part of the record uninteresting with regard to the “CO2 vs temperature” question, and thus that the only period that can say something useful about the “CO2 vs temperature” question is after 1950 or 60 or 70. But that would also be beyond me.
“So please publish this and make sure everybody stops adjusting the data.”
That’s a good point, Anders!
“On a final note, your figure 1 and 4 shows that the adjustments, globally, around 1860 are on the order of 0.4 degrees cooling. That is out of a total change of 0.8 degrees. How you can say that this is minor is beyond me. The cooling pre-1900 looks to be between 0.4-0.1 degrees, which is 10-50% of the entire change.”
Stevenson screen invented in 1864. It seems very probable that the temperature records from earlier are affected by inadequate sheltering. Many of the stations were located in cities as well. Homogenizing methods adjusting those records show that these methods do what they are supposed to do. Fix inhomogeneities.
“Homogenizing methods adjusting those records show that these methods do what they are supposed to do”
No. Homogenizing methods adjusting those records MAY show that these methods do what they are supposed to do. The point is know if a technique works or not.
Robert, Zeke or Steve, could you elaborate on this? Was it inadequate sheltering, the warm city dwellings (urban heat island?). How do we know these things account for what amounts to half of th entire warming?
“Anyways, you have now demonstrated that making adjustments to the record does not contribute anything. So please publish this and make sure everybody stops adjusting the data. You have demonstrated, according to yourselves, that the raw data is more than good enough.”
You wont like the answer if that approach is taken with
A) SST ( adjustments cool the record)
B) UAH ( adjustments warm the record )
Why would I not like the answer, Steven?
Steven, you have explicitly stated several times that adjustments do not not make any difference compared to raw data, that is they do not alter the overall conclusion. Do you still think adjustments are necessary, and if so: why?
“Why would I not like the answer, Steven?”
Because he needs to relegate you.
I am not sure if joking is regarded as “relevant” but I take the risk:
I the reason for the hiatus that they have run out of adjustments?
@ tunka
“I the reason for the hiatus that they have run out of adjustments?”
At least they have been able to demonstrate unambiguously that, as claimed by the experts, at least half of the ‘global warming’ is in fact anthropogenic.
The ACO2 connection to ANY empirical climate variations remains a bit more nebulous.
Zeke,
Why do you use anomaly data rather than measured T?
If the reference period is cooler than it should be, other dates are adjusted hotter – and vice versa.
Further, nobody knows pre-satellite if the reference period is cooler than it should be, because the land temps in particulare are heavily derived from the same original observations. The disagreement that exists between sets is large and man-made.
You have done some good approaches like the scalpel and a long reference period and kriging, but still there are fundamental problems. One is, what to do when you select a temperature to (say) do a SB calculation raising to the power of 4? Do you consider your adjustments are closer for conversion to credible absolute or not? Relatedly, how do we use absolutes from other data sets that can change monthly? Imagine publishing monthly corrections in the jounal that published your paper. It is almost a truism, BTW, that an homogenisation adjustment will change the prior trend.
Another feature that looks odd around 1940. On your first graph, red highs predominate 1940-60, blue highs 1920-40 and vice versa for lows. Did your data study cast additional light on the 1940s problem?
I could go on for hours, but it would emerge that I am old-fashionedly in favour of total avoidance of adjustments in favour of raw data sets with detailed metadata for the region chosen for a study so the investigator can correct what is needed and explain why.
Why concentrate on Berkely Earth? The most often quoted data set is Giss and it is very clear that there have been massive adjustments top this data set.
One only need look at an article by James Hansen from 1999 where he shows, from the GISS data set, that the USA had cooled since the 1930’s up until that time. Look at the USA graph: http://www.giss.nasa.gov/research/briefs/hansen_07/
Now, compare that with the GISS data set for the USA today and tell us all why it has changed to warming and we are now getting “hottest day evah!!!”. More word, it is fraud.
Matti, good question. The Booker article is about GISS and GHCN.
Sorry Paul.
My link to “others” was supposed to go to the Watts
articles on Berkeley
http://wattsupwiththat.com/2015/01/31/saturday-silliness-the-best-adjustments-of-temperature/
Your hyper sensitivity shows in this comment/link. The Josh cartoon posted at WUWT is a generalization of the Paul Homewood anaylses on GISS. ‘BEST ADJUSTMENT evah!’ refers to WARMEST YEAR evah, another of his cartoons on the Gavin Schmid 2014 pronouncement that turned out to have a 32% chance of being right. BEST quite appropriately did not make that error.
Rud,
Schmidt actually had the gall to say in email that it was “likely” – likely, in quotation marks – “It is *likely* to have been the warmest year for the planet”, which indicates by NOAA literature 66.7% to 90% probability, when it was only a 38 % probability – therefore “more unlikely than likely”.
Schmidt used quotation marks in his email to say “likely” . NOAA literature uses the quotation marks to say:
66.7%-90% – “likely”
Schmidt accusing Professor Curry of making things up is the height of dishonest climateballer-talk.
Because Berkeley Earth and GISS (and GHCN for that matter) produce nearly identical results globally. Berkeley serves as a useful independent test (and, as it turns out, a validation) of the approach taken by other groups.
http://www.yaleclimateconnections.org/pics/0115_ZH_Fig1.png
This is complete nonsense Zeke. The question was regarding the USA temperature record which you have not and cannot address.
Ok, Berkeley Earth gets U.S. results nearly identical to NCDC/GISS.
https://curryja.files.wordpress.com/2014/07/slide1.jpg
You still have not addressed the issue. Please take the time to look at the GISS graph in the 1999 article from James Hansen that I linked to. Then compare it with GISS TODAY.
Looks like Steven Goddard also noticed what I did and created a nice overlay for you:
https://stevengoddard.files.wordpress.com/2010/10/1998changesannotated.gif
*crickets*
Zeke Hausfather | February 10, 2015 at 10:56 am
Not necessarily, it just shows both methods have similar results, but that could be that both have the same flaw. I would be pretty surprised if most of the surface series didn’t all augment the analysis by adding modeled surface temps. They all homogenize the data, they all infill missing surface area.
But I don’t think the derivative of station temp is indicative of a gradual forcing, but appears to be dominated by large short period regional swings. Other than these, temps dither around zero.
So the take-away message is that BEST cools the past before 1900 by about 0.4C, cools the 1900-1940 period by about 0.1C, warms the 1950-1990 period by about 0.05C.
I missed seeing this description in all of the BEST papers. Why wasn’t the impact shown?
In addition, what are the possible reasons that a breakpoint algorithm is going to vary so systematically through time? In theory, it should not contain a time function.
What did the weather observers do in 1880 that recorded temperatures 0.4C higher than they really were (and that would be the average of all weather observers across the whole planet)? What changed in 1950 that made them record temperatures that were 0.05C below what they really were (again, all of them or the weighted average of them)?
>”As we will see the “biggest fraud” of all time and this “criminal action” amounts to nothing.”
In your dreams. One problem (of many) is the instances where the different adjustments to the same raw data don’t match just around New Zealand and Australia (let alone elsewhere) i.e. NIWA vs GISS vs BEST (NZ) and BOM vs GISS vs BEST (AU).
I’ve already posted the NZ example upthread of Gisborne Aero NZ (in moderation at this time) where GISS adjusts but BEST doesn’t and here’s a couple in OZ:
BEST make an adjustment to Rutherglen Research 1980:
http://berkeleyearth.lbl.gov/stations/151882
BOM doesn’t adjust 1980 (page 24):
http://www.bom.gov.au/climate/change/acorn-sat/documents/ACORN-SAT-Station-adjustment-summary.pdf
GISS neglects Rutherglen Research entirely:
http://data.giss.nasa.gov/gistemp/station_data/
GISS make no adjustment at all to Alice Springs:
Raw after removing suspicious records
http://data.giss.nasa.gov/cgi-bin/gistemp/show_station.cgi?id=501943260000&dt=1&ds=13
Adjusted
http://data.giss.nasa.gov/cgi-bin/gistemp/show_station.cgi?id=501943260000&dt=1&ds=14
BEST make 8 adjustments:
http://berkeleyearth.lbl.gov/stations/152286
Some BEST adjustments correspond to BOM adjustments, some don’t (page 9):
http://www.bom.gov.au/climate/change/acorn-sat/documents/ACORN-SAT-Station-adjustment-summary.pdf
Just a few examples but certainly not the only ones. Another problem is the comparison of methodologies e.g. Auckland NZ, NIWA vs NZCSC vs BEST vs GISS:
http://www.niwa.co.nz/sites/niwa.co.nz/files/import/attachments/Report-on-the-Review-of-NIWAas-Seven-Station-Temperature-Series_v3.pdf
(No adj for UHI/Sheltering)
http://www.climateconversation.wordshine.co.nz/docs/Statistical%20Audit%20of%20the%20NIWA%207-Station%20Review%20Aug%202011.pdf
(Adj for UHI/Sheltering. This series now in the literature – De Freitas, Dedekind, and Brill, 2014)
http://berkeleyearth.lbl.gov/stations/157062
(No adj for UHI/Sheltering)
http://data.giss.nasa.gov/cgi-bin/gistemp
/find_station.cgi?dt=1&ds=13&name=auckland
(A hilarious dogs breakfast – Auckland airport in 1879?)
The respective methodologies just do not stand up to comparison.
Richard, when you say GISS make no adjustments to Alice springs, that is slightly misleading. GISS start from the GHCN adjusted data, in which virtually flat raw data has been adjusted into almost 2 degrees of warming:
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/5/50194326000.gif
So what you describe as ‘raw’ is not raw at all.
Alice Springs is one of the sites Paul Homewood discussed quite a bit at his notalotofpeopleknowthat blog 2 or 3 years ago.
Paul, it doesn’t matter how many times you raise the point; ‘raw’ means what ever Steve wants it to mean.
Paul Matthews
>”Richard, when you say GISS make no adjustments to Alice springs, that is slightly misleading. GISS start from the GHCN adjusted data, in which virtually flat raw data has been adjusted into almost 2 degrees of warming:”
Yes correct Paul (good someone is paying attention). But for other stations GISS adjust the GHCN data too (i.e. GHCN is as “raw”) e.g. Gisborne Aero upthread i.e. there is no consistency.
Paul graphs the timing of GHCN (not GISS) adjustments to Alice Springs back to 1940:
https://notalotofpeopleknowthat.files.wordpress.com/2012/03/image35.png
From this post:
https://notalotofpeopleknowthat.wordpress.com/2012/03/26/alice-springsa-closer-look-at-the-temperature-record/
Major adjustments 1975 back to 1970, down and back up a little. Major adjustment 2010.
Now compare that to the BEST adjustments:
http://berkeleyearth.lbl.gov/stations/152286
1952 treated differently. 1970 – 1975 treated differently. GISS misses 1979 and 1990. 2000 – 2010 treated differently.
The BEST adjustments only make a 0.28 C/century difference right back to 1880.
The GISS adjustments make a 2 C difference just by 1940.
The respective final datasets may look vaguely similar but they certainly are not in detail.
Should be “The [GHCN] adjustments make a 2 C difference just by 1940.”
Yes Paul, with GHCN we are dealing with one level of adjustments. With GISS we can be dealing with one level (GHCN) or two levels (GHCN + GISS).
It’s nightmarish. At least with BEST there’s only one level but they still don’t adjust for UHI/Sheltering e.g. Auckland Albert Park, and overdo it in Australia compared to BOM e.g. Rutherglen 1980. If they did correct for UHI/Sheltering BEST would corroborate the NZCSC Auckland series but not NIWA’s.
Where BEST comes unstuck in NZ is kriging from far-flung places when there is data at hand e.g. Hamilton in the Waikato province is compiled from data in the Auckland province (e.g. Albert Park) and Bay of Plenty Province (e.g. Tauranga). Neither of which is similar to Waikato climatically because Waikato is hinterland whereas Tauranga is on the Pacific coast and Albert Park on an isthmus between harbours on Tasman Sea side and Pacific ocean side of the North Island NZ. Oddly, BEST overlooks NIWA’s Ruakura Research station right at Hamilton which forms part of NIWA’s Eleven Station Series (11SS).
Probably obvious but BEST’s Hamilton series is nothing like the Ruakura Reseach series situated in Hamilton.
Set NZ to Zero.
calculate the global average.
See? it didnt change.
Why, because NZ is not the world.
Let me extend your argument:
NZ fiddle doesn’t matter. Not the world. (land)
Aus fiddle(Alice Springs, Darwin, Rutherglen) doesn’t matter. Not the world.
US fiddle (GISS 1999-2011 upthread) doesn’t matter. Not the world.
Central South America fiddle (Homewood) doesn’t matter. Not the world.
Arctic fiddle (Homewood) doesn’t matter. Not the world.
Europe fiddle (Steirou) doesn’t matter. Not the world.
Pretty soon you run out of world.
Love your work R.
[1] The assertion that BEST chose an automated method in order to answer skeptics’ is a non-sequitur.
[2] Of the data sources available, BEST is comprehensive and readily accessible. BEST deserves kudos for this. Booker and Delingpole started by questioning the records in Paraguay using Homewood’s work with GISS. Why is BEST answering? The questions were not directed at BEST.
[3] The questions that do point at BEST show BEST to have performed the same adjustments as the other data keepers to local stations. How does showing the global land record answer these questions?
[4] BEST labels station ‘moves’ on their charts for local stations. This was used by Ed Hawkins to imply corrections made in GHCN were not without basis. But ‘station moves’ in BEST are not really ‘moves’ – the term is more a form of short-hand. How many people know this? Ed Hawkins doesn’t know this. Even Carrick doesn’t
[5] With the current episode, listening to BEST, Nick Stokes, Victor Venema et al – it is clear the data keepers have abandoned defending their local station products.
[6] How do you do this:
Show a graph with adjusted and unadjusted global temperatures and say ‘they look the same so it’s all good’
and
Call the guy who’s comparing adjusted and unadjusted local temperatures to say ‘they don’t look the same so it’s not good’, an idiot?
Even if you have your reasoning lined up, if your best argument is from consequence, you’ve half lost the game. The same questions will keep popping up (and making headlines).
[1] The assertion that BEST chose an automated method in order to answer skeptics’ is a non-sequitur.
No.
1. people complained ( remember climategate) that CRU adjustments
were bogus, and not transparent. They were accused of fiddling
the data. We wanted to remove the human element.
2. people complained that hansen was fiddling the data and wanted to
see how he did it.
3. people complained that NOAA had their thumb on the scale.
So we took a “hands off” approach. Not a human sitting there subject to bias adjusting series. But rather an algorithm that asks “what is the best estimate”
[2] Of the data sources available, BEST is comprehensive and readily accessible. BEST deserves kudos for this. Booker and Delingpole started by questioning the records in Paraguay using Homewood’s work with GISS.
Why is BEST answering? The questions were not directed at BEST.
Because Watts and others dragged us into it.see my link.
Because ONE APPROACH to verify adjustments is to use different
data and different method. Its called replication, as opposed to reproduceablity. Standard stuff. shame on you.
[3] The questions that do point at BEST show BEST to have performed the same adjustments as the other data keepers to local stations. How does showing the global land record answer these questions?
The SCIENTIFICALLY INTERESTING question is this.
what do adjustments do to the GLOBAL average.
That’s important because core aspects of AGW rest on the
GLOBAL average. For example. would Curry and Lewis CHANGE
if you used only raw data.
[4] BEST labels station ‘moves’ on their charts for local stations. This was used by Ed Hawkins to imply corrections made in GHCN were not without basis. But ‘station moves’ in BEST are not really ‘moves’ – the term is more a form of short-hand. How many people know this? Ed Hawkins doesn’t know this. Even Carrick doesn’t.
Yes the term is a short hand. People who go through the code know.
users who ask me, I mean people of good faith who want to use the data
who ask me.
[5] With the current episode, listening to BEST, Nick Stokes, Victor Venema et al – it is clear the data keepers have abandoned defending their local station products.
Huh. We’ve been saying for a long time that if you want to look at small spatial scales, its best to start with the raw data. I spent months with Robert Way doing special versions of Labrador. REAL USERS not internet keyboard jockies know this.
[6] How do you do this:
Show a graph with adjusted and unadjusted global temperatures and say ‘they look the same so it’s all good’
and
Call the guy who’s comparing adjusted and unadjusted local temperatures to say ‘they don’t look the same so it’s not good’, an idiot?
very simple. I ask the question.
What Key paper will change if I use raw data?
Put it this way, Will Nic Lewis suddenly run out and say
“using raw data changes everything!!”
will Mann suddenly redo his science
will ANYBODY who has done key science redo their paper because
raw data will change their science.
That is one reason why we had such a hard time getting published.
People forget that science works by FALSIFYING. so what key finding
will change if you change the global series by .05C? No key finding.
Its not interesting to the CORE science. It’s interesting to tecchnicians.
Even if you have your reasoning lined up, if your best argument is from consequence, you’ve half lost the game. The same questions will keep popping up (and making headlines).
And people like you will support the accusations of fraud.
I’ll ask you again.. what I asked you on twitter
How much do I have to cool the record to get you to stop supporting
the accusations of fraud with your writing.
Wow! You are certainly working hard today. My hat is off to you. I have read all the way down to here, but now I have to stop.
matthew I read from bottom up
Starting right out working on the bottom can be a time saver.
shub:
Actually I am fully aware of this. It’s only been explained a dozen times now.
[1] The reason why BEST was initiated is a non-sequitur to current issues.
BEST tries to position itself as having addressed and successfully solved some of the problems with the earlier methods. It hasn’t. With construction of a global average there are only two pieces of data – temperatures and meta-data. BEST is stuck on problems of non-independent data just as every other method is – of trying to glean non-temperature information from instrument series.
BEST attempts to solve some of these problems. It includes a large number of stations. It follows an arms-length approach to the data. But these come with their own limitations: chopping up station segments raises the question of how to align them back together and choices have to be made. The constraints simply pop back in again into the machine.
[2] Watts did not drag anyone into anything. Watts nearly completely stayed out of the whole thing and has been dragged in by you and by the excitable Venema. Publishing a cartoon by Josh is ‘dragging’? Josh’s cartoon plots show NCDC data and he writes:
“There has been much discussion recently about the adjustments made to past temperatures: see Paul Homewood’s excellent posts on Paraguay, Bolivia and around the world; also from Shub; Brandon at WUWT and on his own blog; and a very readable summary by James Delingpole. All very interesting.”
You got ticked off by this?
[3] What’s ‘scientifically interesting’ is a meaningless metric. Many a beautiful theory butchered by a cold fact. Sometimes the most boring, mundane things are the most interesting. I find it scientifically very interesting that scientists think they can adjust a local record toward an average in the process of calculating the average.
Core aspects of AGW rest on the global average. Linear trends in the global average tell people what the average is doing. Having artificially small error bands inflates confidence in false trends and causes scientists to see trends where there are none. A little error would make for a large blessing. It would protect the core aspects from activist oversell and the subsequent backlash.
[4] on station moves: “Yes the term is a short hand. ”
Then don’t label your website graphs with a red diamond labeled ‘move’. It’s not a move if you don’t know for sure. Don’t know it to be? Don’t call it to be.
‘People of good faith’
So you say you display a label outside for public consumption – ‘move’. The people of good faith get a different answer. I think this was a very bad move by BEST. Its moves are just another family of breakpoints, i.e., artifices of a chosen methodology.
[5] “We’ve been saying for a long time that if you want to look at small spatial scales, its best to start with the raw data. ”
Is that so?
This is Tim Osborn of the CRU on local station data:
“It turns out that you can get a good global temperature picture this way, … though the reliability at regional scales will be poor”
Nick Stokes says:
“In fact, the adjusted files that appear on the web should probably be held more internally. They really are a collection of regional estimates, using the station names for convenience. ”
[6] I don’t think you understood the point. People are right to be surprised that raw stations when adjusted look completely different, because you are out to convince them the adjusted global average look rather the same as the raw.
Could someone on the BEST team comment on the quality of the “raw” data? How was it pre-processed before BEST got it? How much of it is really just the numbers recorded by the station keeper and how much already modified in any way?
I would still like a simple answer relating to the Phil Jones, statement in the emails, that he had: ‘dumped the original data to save space at CRU’s new building… When, how (burned, shredded, landfill…), also a manifest of what was collected. A simple statement for the sake of history. Until there is an answer to this question, this is all just an ongoing argument over the best looking pieces of garbage. If we don’t get an explain of the destruction we will never understand anything. It is the root of this error tree. The source of all these numbers we see today. How much was there? How was the data transferred from paper to the data sets?
Sure, we use only raw data. It may come as a surprise to some, but Phil Jones is not actually the curator of the world’s climate data. Rather, he kept copies from individual national MET offices.
Do you know just what was “dumped”? In the 80’s the press had given the impression that the world was putting all their records into a merged data set. CRU was a large part of this effort. Then in 2009, he said what he said… Yet, I have never been able to find out just exactly what is was, that had been dumped. Where could I find out? Thank you, Zeke.
Something like ‘Phil Jones, had 47 tractor trailers full of various written weather records from 1765-1972, burned at Lands End.” It is a simple question that needs an answer. No more trouble than that.
What records did Phil Jones, ‘dump’?
How much was there?
How was “it”, disposed of?
These are all simple, direct questions. The answers should all be easily addressed in a short statement. It does not require a consensus. Any reasonable person would freely give the facts for what they are, if known. Obviously I am not trying to steal the work, since it has already been valued as trash and dumped somehow. If a person is unable to get satisfactory answers to simple questions like this, what is the point of your whole exercise?
Let’s see now…
Phil Jones,… it was taking up to much space, whatever it was.
Eli,…his watch got busted.
Steven Mosher,… you are not a user like me.
Tony B.,.. Who, do you MET?
Zeke,… You don’t know that Phil Jones, is not actually the curator of the world data.
You need to be a physicist or a small child to understand the invisible.
Everyone can see that… it’s as plain as the noses on your back.
When you saw ‘raw data’, do you mean that you use the data that was recorded in the original recordings for each station or data from individual stations that has already been adjusted?
Recorded in original recordings. No adjustments made prior to data intake. No pre-homogenization (or even quality control when possible, though some data intake streams are pre-QCed).
I gave Steve a link a while back to the pdf’s of some US stations, signed off by the State official. The numbers in the State record did not match the ‘raw data’ that was used by BEST. On challenge Steve resorted to sophistry as to the meaning of ‘raw’.
actually Doc you compared the wrong stations.
there is no doubt that you will find paper records that disagree with electronic records. You will find multiple records for the same site.
you will find sites with multiple sensors each reporting to different agencies.
you will find paper records that are later ammended because of entry errors.
There is one school of thought that says.. if we study all these documents then we can recover the “truth”
There is another school of thought that says.. we have no time machine.
we cannot go back in time to verify.
All we can do is use the data we have.. check for consistency, and give our best estimate.
When you find a way to verify that a paper record is the absolute truth.. lemme know.
Mosh
So in essence the temperature number records are as anecdotal as the written records?
I had hoped that you would refute the specific claims made by homewood, delingpole and booker but to date that hasn’t been addressed. Are you able to refute their claims in a straightforward manner that would convince those who were uncommitted ?
Tonyb
tonyb, Nope, they aren’t anecdotal, they are extremely accurate on a “global” scale.
“% Estimated Jan 1951-Dec 1980 absolute temperature (C): 8.70 +/- 0.06
Estimated Jan 1951-Dec 1980 monthly absolute temperature (C):
% Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
% 2.68 3.28 5.36 8.38 11.41 13.58 14.46 13.96 12.14 9.29 6.15 3.72
% +/- 0.07 0.08 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.06 0.07 0.06”
You can see from Berkeley Earth they they have the global mean real, honest to goodness, temperature down to 8.70C +/- 0.06 C.
When they did the combined land and ocean product I am sure that put that global real surface temperature in C degrees +/- something somewhere, I just can’t find it yet.
Tony
The nice article about Lamb on WUWT has a certain relevancy about efforts to understand the historical record. I gained a greater appreciation for his work. Everyone is doing what they believe is right but it still falls short.
Lamb was correct when he said you cant figure out the signal from CO2 if you don’t know everything else that may be going on.
Just not the original recordings in Rutherglen in Australia before 1965, right Zeke? Cos like, according to BEST Rutherglen just didn’t exist before 1965, right? Quality control control isn’t BEST’s strong suit.
“Mosh
So in essence the temperature number records are as anecdotal as the written records?
No for statistical purposes I would treat them as equal.
Then I might test trusting one more than another.
Or I could sample the written records, find that they are 99.9% the same
as the digital records. and then doc would accuse me of fraud.
simple
I had hoped that you would refute the specific claims made by homewood, delingpole and booker but to date that hasn’t been addressed. Are you able to refute their claims in a straightforward manner that would convince those who were uncommitted ?
This post. read it. Adjustments are not a scandal or hoax.
1. its all open
2. the algorithms are tested.
3. they will warm some records
4. they will cool some.
I need to prove that Im not a criminal, nice inquisition.
Steve,
You do not have to prove your innocence, it is up to others to prove that you are guilty.
some poetry, author unknown
Some folks will always be with you
Some you can never understand
Black Sabbath never was the same
Since Ozzie left the band.
Could you get paid less and appreciated less for all that you have done?
> Black Sabbath never was the same
> Since Ozzie left the band.
And vice versa.
My biggest concern with the adjustments is that they tend to demote the Dust Bowl era from temperature primacy. The future climates the Warmist Cult members swoon over are benign compared to the harsh conditions in the southern and western US in the ’30’s.
We see the same nonsense with Sandy vs the Long Island Express of ’38: Sandy was a butter knife smearing across the Northeast; the LIE was a scalpel that carved off chunks of Long Island and remade the Long Island Sound. Had the LIE hit instead of Sandy, the death toll likely would have been in the thousands, not the 124 who lived in a mostly bucolic Long Island.
I don’t see fraud (never did), and I appreciate the Berkeley-ite’s transparency, but I am still concerned about the [unintentional] revision of history it represents.
James Hansen previously acknowledged the Dust Bowl Era as USA’s hottest….. my, how did that change?
https://stevengoddard.files.wordpress.com/2010/10/1998changesannotated.gif?w=500&h=355
https://stevengoddard.wordpress.com/hansen-the-climate-chiropractor/
I am a profane.
I am puzzled by the fact that the effect is significantly larger in the US than, say, in Afrika. It is all fine to say that the black curve more or less cancels, but I would think that it is not by accident (I mean the result of randomly distributed causes) that the US deviations are large, there must be some
reason behind it. Could you, experts, explain to us profane which is the dominant mechanism giving this result?
Thank you so much
Bacpierre
Hi Bacpierre,
Reasons for large US adjustments are discussed at length here: http://www.judithcurry.com/2014/07/07/understanding-adjustments-to-temperature-data/
More piffle from you Zeke. I have shown how the data changed from 1999 to what it is today for the USA. Are you saying that Hansen was WRONG in 1999 to say that the USA had cooled since 1930? More like that particular notion no longer fits the warmists’ desperate meme, considering that ALL we have had was a minor 0.3 degrees warming between 1978 and 1998. For this we are supposed to run around squarking “Doom, doom!!!!”….?
So Robert, Zeke and Steve, you have a nice determination of global temperature, by area and globally, that covers >100 years.
Now you have to test the calibration. You have stated here and in the past that you treatment of data has emulated the actual temperature field, in spite of station moves and in spite of sensor changes and the way the sensors were read. It is now time to test your hypothesis to destruction.
You are aware of the type of sensor, and the housing of the sensor, used in the USA 100 years ago. You know exactly these sensor were read and how the data was recorded. You must, a prior, describe the minimum number of stations and their spacial distribution required to generate a temperature field.
You reproduce, exactly, the type of stations used 100 years ago, and distribute them in adequate sites and monitor the temperatures that the reproduction thermometers record. After three years you compare your BEST derived temperature field with that of the actual temperature field, recorded by historically accurate stations.
The data of the historical sites is to be drawn from the actual paper images or authorized copies of the original data logs, and not from the easily accessible digitized datsets.
The historically matched thermometer field is compared to that of 100 years ago, to give us the average monthly max, min and (max+min)/2, recorded in the same way, under very much the same conditions.
You compare the differences your algorithm manages to compare modern sites/sensors vs historic sites/sensors, having a prior, stated how these two types of data will be assessed for statistical significance.
I think you have the line shape, but not the calibration.
Let’s not forget that modern electronic thermometers will instantly record brief fluctuations in temperature that bulb thermometers would never record…. such as that hot breeze momentarily coming across from the concrete car park. All modern high temperatures above 30C need to be dropped down by at least 2C.
Matti,
I’m only familiar with airport-based systems, but these all use a 1-minute average for individual measurements so any brief fluctuation would need to last at least that long.
I believe they compare averages like 30 min or one hour to avoid spurious peaks. Still electronic instruments have different biases, for example electronic recorders would be limited at really low temperatures like -70C because batteries tend to drift. On the whole I doubt it is much of a problem. UHI appears to be more of a water cycle issue.
Cap’t, there is a big difference between knowing and believing; we can all guess as to what biases this or that instrument will give, but until you actually perform an experimental determination, it is just bias.
“I’m only familiar with airport-based systems, but these all use a 1-minute average for individual measurements so any brief fluctuation would need to last at least that long.”
Such a pity that you cannot see the logical fallacy with your statement. Please take the time to think further upon it.
Matti,
Nope, you’ll have to enlighten me. But please note that all the systems I work on are designed to WMO climatic standards, in order to avoid exactly the ‘brief fluctuations’ you mention.
Sorry Jonathan, but any “1-minute average” will include and fluctuations and can read higher. Are you aware that with UHI there can be spikes of more than 10 degrees??
Matti,
Ah, now we’re getting into a different kettle of fish. The systems with 1 minute averaging provide very smooth temperature records, I am yet to see a sudden fluctuation or jump within daily records.. Now in theory as you suggest these are bundled up into the 1 minute averages, but if that were a major factor I would still expect to see less stable traces. Which I don’t.
As I understand it, the adjustments that are applied to readings between bulb thermometers and electronic ones are based on extensive field trials, which would of course then allow for exactly the effect you mention. I haven’t looked into the details of the field trials and adjustments, if anyone has some data on it I would be interested.
I would expect the effect of sudden fluctuations to be more pronounced at stations where UHI is a problem, and UHI does concern me but pending release Anthony Watts’ work-in-progress paper on this I don’t have any robust data to go on. Without going into details, Mosher’s previous defences of BEST against the problems of UHI have passed the smell test at the time.
So Robert, Zeke and Steve, you have a nice determination of global temperature, by area and globally, that covers >100 years.
Now you have to test the calibration. You have stated here and in the past that you treatment of data has emulated the actual temperature field, in spite of station moves and in spite of sensor changes and the way the sensors were read. It is now time to test your hypothesis to destruction.
You are aware of the type of sensor, and the housing of the sensor, used in the USA 100 years ago. You know exactly these sensor were read and how the data was recorded. You must, a prior, describe the minimum number of stations and their spacial distribution required to generate a temperature field.
1. Actually we dont know for sure ANY of that.
2. We know the type of sensor reported in a document. Not the
actual sensor in the field.
3. We know the type of shield reported in a document, not
the condition of the actual sheild in the field
4. We know the INSTRUCTIONS to the observer, we dont know
that he followed them
you are not skeptical ENOUGH.
You reproduce, exactly, the type of stations used 100 years ago, and distribute them in adequate sites and monitor the temperatures that the reproduction thermometers record. After three years you compare your BEST derived temperature field with that of the actual temperature field, recorded by historically accurate stations.
Actually a better approach since our field is a prediction of the temperature AT PLACES WHERE IT WASNT RECORDED, is to
place new sensors at random places and test the prediction.
The standard error of prediction on a monthly measurement is something
on the order of 1.5C. The other way to do the same thing is to hold out
stations. Been there done that.
#########################
The data of the historical sites is to be drawn from the actual paper images or authorized copies of the original data logs, and not from the easily accessible digitized datsets.
You realize that the paper copies are not QC’d
you realize that a single site can have multiple records
However, if you want to disprove the work we did, go get the paper copies.
for the whole world. find the errors. report them. they wont amount to anything.
####################################
The historically matched thermometer field is compared to that of 100 years ago, to give us the average monthly max, min and (max+min)/2, recorded in the same way, under very much the same conditions.
You compare the differences your algorithm manages to compare modern sites/sensors vs historic sites/sensors, having a prior, stated how these two types of data will be assessed for statistical significance.
I think you have the line shape, but not the calibration.
We dont aim at getting the calibration correct. In fact the error on the temperature field is much higher than the error on the anomaly.
Its the trend we care about. its the CHANGE we care about. The absolute
calibration to true temperature.. you’d have to trust way too much stuff that is questionable: basically historical documents
“Its the trend we care about. its the CHANGE we care about”
Why is something so simple so difficult for some to get?
No cigar. You either want to establish the DeltaT or you don’t.
The only way to do that is to emulate the measurements used in the past; all else is arm waving.
That your ‘raw’ and actual ‘raw’ are different and you are uninterested is most reassuring.
I dont know joshua.
Since there is NO calibration record, ACCURACY is going to be hard or impossible to verify.
But we can still get the trend.
shakes head..
Mosher doesn’t trust historical documents. Mosher says recorded history is questionable. This is the nub of climate science – it’s just a theory complete divorce from actual local historical weather.
I’d suggest that the likelihood that any historical record has ever falsified the weather, is somewhere between the zero and zip. Think about this for a moment like a lawyer thinks about evidence. Record A B and C show BEST’s amazing hands-off no human interference algorithm has got the 1930s and 1940s wrong by at least a degree, say. What possible reason would a recorder of history, on any topic, anywhere in the world ever have had to falsify the record of the weather happening at that time? Horse racing meeting records, football game records, cricket game records, Farmer John’s rain fall records etc etc, reports about a school fete, hill climb races, Cinco de Mayo festivals, the literature of the day, paintings by landscape artists, photographs of weddings on Sydney Harbour etc etc, and yes, even the notes of the guy monitoring the weather station at location A, whether it was a Stevenson screen or a glass of mercury on a stick. Every one of those records, wherever they were taken will have noted the weather and there’s no reasonable basis to doubt them. History is just day old+ news.
Mosher thinks we should be sceptical of history. Seriously, mate, read more history, comment less.
I didn’t ask for a calibration record, you are again using language to make the world a more muddled and confused place.
You claim that you are able to construct a temperature field based for the year 1900, in a particular local, on past measurements and will be able to construct a temperature field for the year 2016, in the same local, using the present temperature sensor set up.
I do not believe that the two temperature fields are necessarily comparable, as I have no faith in your temperature field construction. I have no faith, because of your lack of controls. I therefore suggested a control that I believe is valid if one wants to be able to compare a temperature field of the past with one of the present. The control is to run the present day network over a local, and at the same time emulate the type of measurements recorded in 1900. Compare the temperature field of 2016 taken from the existing sensor system to the temperature field generated by max/min mercury thermometers in 1900 era screens.
That you are so intellectually arrogant that you refuse to contemplate the use of internal controls, caution us that anything you generate as a work product must be viewed as biased.
Jonathon,
Averages will always be smooth. There is no teaching a true believer, so just forget it.
Can someone tell me what is to be done with the actual temperature values of the 1930s in the US if they were records, given the modern adjustments that have taken place since then?
Are we to assume that if we stuck the same thermometer out at the right time in the same place in the present, we definitely should have seen a higher record temperature at some point?
I’m trying to reconcile the accepted adjusted increasing temperature anomaly trend with the companion downward trend of reduced “days > 90°”, having peaked in the ’30s (for example, at many stations in West Virginia).
How can any “record” high for an area be accepted if things have moved around and changed since the last time it was that hot…..?
Hi morebrocato,
Original data is of course kept, but you need to be careful with records if your station has moved or your instrument has changed. For example, a typical city weather station was on top of a building roof prior to 1940 or so (a location that tends to be pretty hot!), after which it was moved to a grassy patch on a newly built airport or wastewater treatment plant (where temperatures tend to be a bit lower). In the 1980s the old liquid-in-glass thermometer was likely replaced with a new MMTS electronic instrument, which is more accurate but (unfortunately) tends to read maximum temperatures about half a degree lower than the old mercury thermometer.
The adjustments we are discussing are an attempt to deal with all of these changes (and any station thats been around for 100 years has multiple moves and generally at least one instrument change).
“which is more accurate but (unfortunately) tends to read maximum temperatures about half a degree lower than the old mercury thermometer.”
This is an outright lie! A modern electronic thermometer will instantly record any brief fluctuation in temperature which a bulb thermometer never would…. such as a breeze momentarily bringing all that car park hot air across.
An outright lie?
http://rankexploits.com/musings/2010/a-cooling-bias-due-to-mmts/
http://ams.confex.com/ams/pdfpapers/91613.pdf
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/menne-etal2009.pdf
http://journals.ametsoc.org/doi/abs/10.1175/1520-0477%281991%29072%3C1718%3AEORTCI%3E2.0.CO%3B2
Thanks Zeke!
I feel like I’m still at a little conundrum with this though. Based on your response, is there a dirty little secret that the 1930s ‘should not’ have been as hot, or all these extreme events likely contaminated? Isn’t the lion-share of sites that stretch back that far requiring adjustment in some way? (Time-of-day, Siting change, etc.)
Is it postulated rather that if we maintained what we had back then all the way to the present that we’d have seen even more impressive extreme temperature records?
If either one of these are true, then why not a legion of asterisks applied to extreme max temperature records that exist from 80 years ago? Too much wading in the weeds? Something only for scientists to understand, and not laymen?
Zeke thanks so much for this comment because it reveals how completely misguided BEST and the rest are. So what if the temp measure was on the roof in the 1940s? Doesn’t mean it wasn’t hot. It was still hot in the 1940s. BEST has adjusted the actual heat away.
I find remarks like this baffling:
I don’t know too much about the mechanics of the temperature sensors being discussed. I can accept the possibility someone might show what Zeke Hausfather said about them is wrong. What I can’t accept the idea he is lying about it. Even if I thought Zeke a dishonest person who’d be willing to lie about such things, why would he lie about something so easy to check?
Another remark I find baffling is:
I’ve said quite a few things in blog posts. I believe some of them have been somewhat important. I don’t understand why them being said in a blog post would somehow mean they shouldn’t matter. Blog posts are just a medium. Like any medium, they can convey good material or bad material. They can convey high quality or low quality information. They can contribute as much or as little as anything else.
What I find most baffling about remarks like these is they have nothing to do with positions. The same sort of remarks can be made by anyone, about practically any argument. What’s the point of making them? Why say something anyone could say about anything? How does it contribute to anything?
I don’t get it. I wish people, no matter what side of the argument they may be on, would reject things like these.
Brandon Shollenberger: I wish people, no matter what side of the argument they may be on, would reject things like these.
I support your defense of Zeke Hausfather. I know that sounds pretentious, tendentious and such. But really that guy went too far.
I am sorry for Brandon and others who seem incapable of comprehending the presence of absolute piffle.
What a wonderful rebuttal of Booker and company.
It is oh so compelling.
However there is one little problem with it, which everyone so far has been too polite to mention.
The problem is that the results you present are using BEST algorithms for the RAW Global Temperature Trend.
We do not know what affect that has on what you insist is the RAW data trendwise.
But of course we do know because Mr Hausfather made a very big mistake in the post on WUWT rebutting the original Steve Goddard posts.
It was entirely glossed over at the time, but I was totally shocked by it, let me remind you of what the GHCN RAW data REALLY looks like according to Mr Hausfather.
http://rankexploits.com/musings/wp-content/uploads/2014/06/Averaged-Absolutes.png
Now does this graph presented by Mr Hausfather look anything like the RAW data trend presented by BEST?
What happened to the 1.5 Degree C step up in temperature around 1950, what happened to the 1.5 downward trend after that, what happened to the 1.8 degree C step up around 1992 and the subsequent almost 2 degree drop around 2000?
Is Mr Hausfather now going to deny the graph of GHCN Raw data that he presented to rebut Steven Goddard?
Hi A C Osborn,
That graph is an illustration of why simply averaging raw temperatures when your network composition is changing over time is not a viable way to estimate a global temperature. Using anomalies (or the Berkeley approach, which is effectively similar) avoids that issue. See this discussion for an explanation of the basics: http://rankexploits.com/musings/2010/the-pure-anomaly-method-aka-a-spherical-cow/
However, Goddard’s misguided averaging of absolute temperatures is somewhat immaterial to the discussion at hand.
Sorry, you do not get out of it that easily.
If that is what the simple RAW data looks like when it is averaged, then THAT IS WHAT IT LOOKS LIKE.
Carrying out Homogenisation, Dicing & Splicing, TOBS and the Gridding or Krigging or what ever you use has lost the original simple data.
If you showed the original data and the Final Product that would be understandable, but you don’t.
Those upward Steps were produced by something, it was not the change of a few stations, it could not possibly have that much effect. So your work has lost that valuable data.
I pretty much accept what you’ve said here, thanks for the explaination. It does raise some questions for me though. I take it you accept what Homewood has discovered in that Puerto Casado has been adjusted upward by one degree since you didn’t address that specifically. He is also looking at upward adjustments elsewhere. Previously others have pointed to upward adjustments in New Zealand and Australia. Obviously there has been upward adjustments in some places and like you said downward adjustments made in Africa and elsewhere. It apparently results in a net zero affect as you state. So what is the point of doing any adjusting at all? It just seems to raise suspicion and I don’t understand why you do it anyway? It also introduces the question of bias. Since all of the land instrument data bases are controlled by warmests, it doesn’t take much imagination which way that bias would go.
I might also add if the problem with all of the raw data is from the urban heat isle shouldn’t all the data be adjusted downward? If the adjusted data is net zero wouldn’t that mean it is still running hot?
The question you should be asking is this.
Why didn’t homewood present any cases where we cool the record?
GISS cools the record as well. Why did he avoid those?
Next:
So what is the point of doing any adjusting at all? It just seems to raise suspicion and I don’t understand why you do it anyway? It also introduces the question of bias. Since all of the land instrument data bases are controlled by warmests, it doesn’t take much imagination which way that bias would go.”
1. We did adjustments because people thought GISS and others were doing diabolic things. What we showed is that our global record matches there global record. And using ALL the data, adjustments amount to very little
2. We do adjustments because some people want the best estimate we can come up with.
3. We do adjustments for the same reason folks wear glasses.
“Since all of the land instrument data bases are controlled by warmests, it doesn’t take much imagination which way that bias would go.”
Wrong. You are aware that the Ocean database is “controlled by warmists” and the adjustments are a LARGE cooling.. 70% of the earth data is COOLED by adjustments and 30% ( the land) is largely unaeffected
Thanks for answering my questions, I am satisfied with that response.
Wow, more misdirection from the Mosh. I am not asking Homewood anything. He doesn’t strut round calling himself BEST.
Or compare the Mr Hausfather USA graph with their USA Raw Trend
https://wattsupwiththat.files.wordpress.com/2014/06/goddard-and-ncdc-methods-1895-2013.png
I think this might be the graph you are looking for… https://curryja.files.wordpress.com/2014/07/slide1.jpg
Now the global chart next to the U.S chart looks very weird.
1.The gap between adjusted and raw data is much bigger for the U.S. data.
2.There is no significant gap in the global data after about 1975, while there is still a significant gap in the U.S. data up to the mid 1990s.
3.Looks like the U.S. was on a different planet with a different climate from about 1930 to 1960.
What’s up with this stuff? Are we to believe that the U.S. raw data has required more adjustment than the global data? Sum Ting Wong, here.
The global raw data before 1940 was likely useless.
I gotta agree with Don here. The notion that US measurements require more adjusting than the rest of the world just doesn’t pass the laugh test.
I don’t want to get hung up on the adjustments stuff (for the reasons in my comments down thread) but stuff like this sticks out like a sore thumb.
The U.S. had big systemic adjustments in the 1960s-1990s (time of observation changes, instrument changes, etc.) that were much less common in the rest of the world. Much of this is due to the fact that the U.S. network was largely volunteer-run, while most other countries have the (generally smaller) network run by the MET office.
Again, this is discussed in great detail here: http://www.judithcurry.com/2014/07/07/understanding-adjustments-to-temperature-data/
The U.S data had significantly more adjustment, from the start of the chart 1900.
Most of the global data is sea surface, polar ice caps, sparsely inhabited desert etc., isn’t it? Why should U.S land data need more adjustment than that sketchy data, especially in the earlier years? I will look at your reference to previous discussion.
OK, I get why the U.S. data is adjusted. But the U.S raw data that needed so much adjustment was of better quality than the data from the rest of the planet, with large parts having no data at all. Wasn’t it? Doesn’t this tell us that the data from the rest of the planet, particularly before about 1940, is rather suspect, to put it mildly. I am guessing that data from the U.S. and a few other civilized countries, using some fancy kriging from coastal and island stations, is a better representation of world temperature than the so-called global data. If I cared as much about this stuff as some people do, I would investigate that theory:)
Why are you questioning your own Graph, don’t you believe you could have cocked it up so badly or something?
“But the U.S raw data that needed so much adjustment was of better quality than the data from the rest of the planet, with large parts having no data at all. Wasn’t it? ”
I dont think its settled science that the US has the best network.
it has a lot of stations.
But many of them run by volunteers in the past.
thats why we have TOBS..
“with large parts having no data at all.” I should have said most of the globe having no data at all. No need TOBS on no data.
“I am guessing that data from the U.S. and a few other civilized countries, using some fancy kriging from coastal and island stations, is a better representation of world temperature than the so-called global data.”
I am suggesting that for pre-1940 global temp estimation. You can easily do that:)
You are swatting a lot of mosquitoes today. It’s almost cocktail hour. You can take a break. They will still be here when you get back.
As a matter of curiosity, have there been any attempts to characterize the ‘noise’ in temperature data? I believe the historic record consists mostly of daily high and low integers and that their average is used to construct series for subsequent regression analysis. From the git-go, half the available information is discarded.
Pingback: Climate Denial Empire Strikes Back with Bogus Temperature Story | Climate Denial Crock of the Week
Urban heat effect Zeke 15/11/2012
21% rise since 1985 and 9% since 1960
Homogenisation used to reduce this difference from 1930 USHCN onwards.
NOT TO REMOVE IT.
So current temps from city records incorporate a known 21% urban heat effect but this is OK because you gradually reduce it back to insignificance by 1930.
Hence warming the present by “real” measurements and cooling the past, twice in fact.
Congratulations Mosher, on your Hand on heart statement. ” I checked the figures and could not find a problem with the UHE.”
Guess it depends on what you were checking?
Only half removed prior to 1930 so a second homogenisation done by NASA GISS to remove the other 50% prior to 1930.
Finally the homogenised Rural subsets compared to the homogenised Urban heat islands is “sufficient” to limit the effect of using these homogenised heat islands.
Problems
Zeke, this could be interpreted as you leaving the heat island effect in all the urban data you have used and continue to use.
Homogenising wrong data is not the same as removing it. Alleviating it with rural data strongly suggests you left the heat in instead of removing it,
Doing a second ” remove ” or adjustment pre 1930 suggests that you dropped those pre 1930 temperatures, possibly only the rural ones, but probably both way down.
Finally it makes your comment that the raw and adjusted data are comparable in the past totally laughable. How can there possibly be any similarity with raw and adjusted USHCN figures with such massive, self admitted adjustments.
Hi Hweale78,
Post homogenization rural stations have “a slight but not significantly different from zero reduction in warming over the 1960–2010 period”. See section 4.2 and Figure 2 in our paper: ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/hausfather-etal2013.pdf
Cool, thanks. I do like the idea of comparing variances between models and datasets. People get really hung up on point values alone, but variances and higher order statistics have an important story to tell as well.
One question for the authors, please.
I understand that BEST was designed to deal with global metrics but I wonder about the value of your data on smaller scales. I am specifically interested in the temperature trends in New York State and have argued that BEST data are appropriate to use. Any thoughts?
Hi rogercaiazza,
At a state-level Berkeley Earth provides a high-resolution (quarter degree) dataset that might be of use. Series like Berkeley may not be as useful for an individual city, as non-climatic factors local to the city (e.g. a very real local heat island effect) may have been removed as they are not representative of the larger region.
Here is the quarter-degree US dataset: http://berkeleyearth.lbl.gov/auto/Global/Gridded/CONUS_TAVG_LatLong0.25.nc
Thank you!
Don’t listen to that without testing ir for yourself, BEST can’t even model (and that is what it is doing), the UK as I proved to Mosher on another post. It can’t handle coasts and Islands with prevailing winds and Ocean Currents.
Also Mosher made the most outragous statement that given the Latitude & Elevation he could give the correct Temperature of any location to within 1.5 degrees C with 93% confidence.
When I asked him to prove it he ignored the question.
He also said and I quote
“Steven Mosher | July 2, 2014 at 11:59 am |
“However, after adjustments done by BEST Amundsen shows a rising trend of 0.1C/decade.
Amundsen is a smoking gun as far as I’m concerned. Follow the satellite data and eschew the non-satellite instrument record before 1979.”
BEST does no ADJUSTMENT to the data.
All the data is used to create an ESTIMATE, a PREDICTION
“At the end of the analysis process,
% the “adjusted” data is created as an estimate of what the weather at
% this location might have looked like after removing apparent biases.
% This “adjusted” data will generally to be free from quality control
% issues and be regionally homogeneous. Some users may find this
% “adjusted” data that attempts to remove apparent biases more
% suitable for their needs, while other users may prefer to work
% with raw values.”
With Amundsen if your interest is looking at the exact conditions recorded, USE THE RAW DATA.
If your interest is creating the best PREDICTION for that site given ALL the data and the given model of climate, then use “adjusted” data.
See the scare quotes?
The approach is fundamentally different that adjusting series and then calculating an average of adjusted series.
in stead we use all raw data. And then we we build a model to predict
the temperature.
At the local level this PREDICTION will deviate from the local raw values.
it has to.”
AC.
see figure 3
http://static.berkeleyearth.org/papers/Methods-paper-Appendix-sm.pdf
Mosher, I don’t care about your papers, BEST doesn’t work for the UK, how many other places doesn’t it work for with similar climatic conditions?
And you still haven’t answered my question, when 3 places on the same Lat and elevation vary by over 10 degrees C how can you justify your outrageous statement?
“And you still haven’t answered my question, when 3 places on the same Lat and elevation vary by over 10 degrees C how can you justify your outrageous statement?”
Simple. you forget to remove seasonality. You forget that I’m talking about the average error of prediction. while 3 places may differ by 10 degree, 100000 places will differ by tiny amounts.
It’s Mulligan Stew. What you can fish out of it may not look at all like what you put into it, in fact, it may not be the same thing at all. But, bon appetit!
==================
Steven Mosher | February 12, 2015 at 12:19 pm |
Simple. you forget to remove seasonality. You forget that I’m talking about the average error of prediction. while 3 places may differ by 10 degree, 100000 places will differ by tiny amounts.
JC SNIP, you do not have 100000 places with Weather stations in the whole world let alone on the same Lat and Elevation.
You do talk some absolute crap at times.
2 locations just in the UK 40 miles apart either side of the Pennines with a 10 Degree C temperaure differential on one day.
Your Algorithm could not possibly handle it and niether can your original statement.
AC you still dont get it.
You have an infinite number of places you can check..
Mosher, no it is you who “doesn’t get it”.
Thr=ere is nothing wrong with the original data, but your algorithm will change it.
rogercaiazza, ” I am specifically interested in the temperature trends in New York State and have argued that BEST data are appropriate to use.”
Climate Explorer has a mask that includes states that you can use to compare different interpolation/adjustment versions.
http://climexp.knmi.nl/selectfield_obs.cgi?someone@somewhere
https://lh5.googleusercontent.com/-kkvameZP34k/VNpvnPKYsGI/AAAAAAAAMoA/rd-bXYpIm64/w909-h511-no/Georgia%2BTemperature%2Banomaly.png
This is one for Georgia.
rogercaiazza,
There appears to be a consensus for New York State
https://lh6.googleusercontent.com/-6yc3Q7dxCT8/VNp6RLI5oDI/AAAAAAAAMoU/8yDRLdGn5pE/w911-h495-no/New%2BYork%2BState%2BTemperature%2Banomaly.png
All data sets are problematic and uncertain.
Land/Ocean indices are in same ball park considering MSU,Ocean,Raob.
Homgenization can’t fix UHI.
All temperature trends since 1979 are less than model and IPCC4 predictions.
As has been mentioned, the premise of this article from Berkeley Earth:- “Christopher Booker’s recent piece along with a few others have once again raised the issue of adjustments to various temperature series, including those made by Berkeley Earth, is completely wrong.
The Booker article in the weekend UK Daily Telegraph (still at http://www.telegraph.co.uk, last I saw comments were unprecedented over 20,000) has no reference at all to BEST. It is entirely based on Paul Homewood’s (www.notalotof peopleknowthat.wordpress.com) analyses and observations from NASA NOAA and GISS data.
It’s interesting but not very useful for BEST to provide their own explanations. A full and detailed response need to come from NASA NOAA/GISS.
Note that I linked to one of the others who made an issue of BEST.
Booker raised the issue WRT NOAA, OTHERS ( see the link ) made the issue BEST as well.
read harder; comment less
BEST enters the picture only because Ed Hawkins and others used it to defend adjustments made to a station in Paraguay. I showed their moves were not ‘moves’. Delingpole mentioned my article.
Berkeley et al are welcome to defend the practice of adjustments but no one directly involved them.
Data Adjustments Save Models !
Temperature adjustments are minor compared to adjustments made by space “scientists” to hide the fact that the Sun selectively moves lightweight elements and lightweight isotopes of each element to the photosphere and solar wind.
The curved line defined by data points from analysis of Apollo lunar dirt was converted into a flat, horizontal line to confirm the Standard Solar Model’s prediction: The interior of the Sun must be composed of H and He just like the Sun’s outer photosphere:
http://www.omatumr.com/Data/1983Data.htm
No shub, you and anthony piled on.
and then morano and drudge pulled up a old quote by anthony that this was criminal.
In other words we were getting smeared.. I get mail from users.
cant just stay quiet when people are playing a guilt by association game.
you wouldnt.
here shub
http://wattsupwiththat.com/2015/01/31/saturday-silliness-the-best-adjustments-of-temperature/
we literally enter the picture.
now. I had the common sense to call out bad behavior on the part of Jones and mann in climategate.
yet.. you remain silent. Thanks for helping with the smear job.
I did not pile on and you cannot pin such things on my head.
Ask whoever’s getting worked up about cartoons to take it easy. Not a good time to be getting worked about caricatures.
Zeke,
A comparison of adjusted and raw series has no interest if you do not know the average length of the series. How long is it?
Hi Phi,
Its common to start land temperature series in 1850, as that is a point where there is reasonable global coverage. You can go earlier, but the error bars increase pretty rapidly. Both the raw and adjusted series are calculated over the same period.
In terms of average length, I don’t know offhand, but probably on the order of 20 years or so? We are dealing with 40,000 stations, and while some have nice long records, many are much shorter. However, even if you only use long stations (and ignore the short ones) as folks like NCDC or Hadley do, you get similar results.
Thank you for your reply.
With an average of 20 years, it is quite logical that adjustments are weak or nonexistent. The real adjustments are made between series but are invisible.
Do you know the average length of the series for GHCN, (I think it must be of the same order of magnitude)?
CRUTem use in principle the homogenized data from national offices. These data are generally based on long series, depending on the country probably from 60 to 150 years of average length. Adjustments are fairly steady in these cases and it is about 0.5 ° C per century.
Hi Phi,
GHCN uses only stations with a complete record during their common anomaly period (1961-1990). The average record they use is 75 years long.
Erm, I meant NCDC, not GHCN per se (though NCDC uses GHCN).
Assumes that continuous long records are better.
bad skeptic
I don’t see many fish in this barrel.
All that any of this demonstrates is just how desperate the warmists are.
Some facts:
1. There is no statistically significant difference between the warming rates of the late 1800’s, early 1900’s and between 1978 and 1998 (Professor Phil Jones, CRU East Anglia). Thus there is no “signature” for CO2 “forcing” in the temperature record. It has not and cannot be measured.
2. The planet cooled between 1940 and 1975, all the while that anthropogenic CO2 levels were escalating wildly. Why did it not warm according to the AGW/Co2 hypothesis.
3. 1/4 of the total of anthropogenic CO2 emissions since the beginning of The Industrial Revolution have occurred since 1998, yet the temperature trend since 1998 has remained flat. Why has it not continued to warm?
These three points alone completely falsify the AGW/CO2 hypothesis, let alone other well known factors crucial to the hypothesis.
We constantly hear cries of “Hottest year evah!!!”, yet we are talking in thousandths of a degree and factors which are not statistically significant.
Grow up guys, AGW/CO2 theory is a crock!
Well said Matti Ressler. The 3 points you make clearly falsify the CAGW/CO2 meme and show it to be a hoax. Emphasis on the C in CAGW.
As I understand the AGW/CO2 hypothesis is (as repeated many times by Richard Linden) that “all other things being equal” a doubling of CO2 in the atmosphere will lead to a 1 deg temperature rise. This is a “so what” hypothesis that may or may not be contradicted by your 3 points.
“about 0.2 C per decade trend”
trend since WHEN? You only have since 1950 and there is no way on Earth you can show a 0.2C warming trend since 1950. No way.
That is 64 years sonny, with less than 1/3 of that with any warming trend at all.
There is nothing to falsify, it is piffle.
AGW/CO2 theory is a crock!
ehh….. I’m not there.
Observed trends are all less than modeled, but they’re all positive, also.
The AGW result may be exaggerated ( perhaps negative feedbacks ).
AGW impacts may be insignificant to human and ecosystem well being.
But it’s probably real.
No Lucifer. Unless you wish to contend that the warming of the late 1800’s and early 1900’s were not natural (you are very alone there), then you cannot in any way show CO2 “forcing” in the temperature record.
you cannot in any way show CO2 “forcing” in the temperature record.
Sure you can – temperature trends are all positive since 1979, less than modeled but greater than zero, consistent with theorized increase in radiative forcing. That’s not the end of the story, of course, the atmosphere continues to churn. But zero? No.
“Sure you can – temperature trends are all positive since 1979, less than modeled but greater than zero, consistent with theorized increase in radiative forcing. ”
This is a complete crock Lucifer. You must now contend that the warming of say the late 1800’s was induced by CO2 forcing. You are totally lost with that argument.
To show “forcing” the temperature trend must be greater than natural warming. It’s not.
“temperature trends are all positive since 1979”
Absolute piffle! Temperature trends were positive from 1979 to 1998. We has a WHOPPING warming of 0.3 degrees in that time, for which we are all supposed to run around squarking “The sky is falling…. the sky is falling”. Since 1998 the trend has remained FLAT. You do understand statistical significance do you not?…. like 3 thousandths of a degree is NOT.
You don’t understand statistical significance.
The trends since 1998 do not exclude the “about 0.2 C per decade trend”
So you fail in your falsification attempt.
So try harder
“about 0.2 C per decade trend”
trend since WHEN? You only have since 1950 and there is no way on Earth you can show a 0.2C warming trend since 1950. No way.
That is 64 years sonny, with less than 1/3 of that with any warming trend at all. That was 20 years with a warming trend on 0.15 degrees. From where do you conjure your 0.2 degrees Mandrake?
There is nothing to falsify, it is piffle.
Matti Ressler,
Thanks for seeing the big picture (the dog) while these learned gents study the fleas.
Here is a question for Mr Zeke and Mr Steve from a layperson who graduated many moons ago with NO knowledge of science. All that I have learned I owe to Steve McIntyre and more recently to Climate etc.
I have always felt that BEST had science’s best interests foremost. (Never doubted its accuracy the way I did with GISS/NDCN/NOAA//HADCRU).
What is the value of a global temperature? I am not saying that there isn’t, just that I cannot see it. I see perhaps that after having several 1000’s of years of world data we could start to prove that the world does have an equilibrium system (or not) or if there were a sudden rise of several degrees in temperature…(but we would see and feel that long before there was statistical proof.) But global temp x is made up of 10’s of thousands of local temps. A world where the entire Australian continent, huge chunks of Canada and Africa and the Middle East were in extreme drought with very high temps – that world could have the same GT as a world where Australia et al had cool temps and plenty rain but a whole lot of other places had slightly elevated temps. But averaged out the GT could be the same. Until we have regional/sub continental temps to work with and can make accurate predictions – I cannot see that GT is useful for climate policy.
I am ready to learn….
This is a good point. I’ve never understood the total emphasis on a “global temperature”. The main concern of the world’s population, is there is one, would be how and whether the local or national climate where they are is changing, or is expected to change and why.
It must be possible to produce comprehensive temperature records and trends from the actual reported national temperature data. In the case of geographically large countries e.g. Canada, USA, Russia, Australia, it regional records would be needed as the national climate varies so much.
This approach would not need adjustments, gridding, homogenization, but should as logically and accurately as possible take account of UHI effects of the applicable stations, or perhaps only “rural stations” should be included.
If few national records and trends demonstrate obvious warming, then I don’t see how a “global temperature” could be claimed to show warming.
GT is a political construct
and like politics…all climate is local
“If few national records and trends demonstrate obvious warming, then I don’t see how a “global temperature” could be claimed to show warming.”
It can’t
Well gee. Did we come out of The Little Ice Age or did we not? Was that a natural occurrence? Was it a good thing?
its a single limited metric of a very complex system.
people think its more important than it is.. my view
Zeke – many thanks for addressing the questions on homgenisation.
Have “Steve Goddard’s” contentions that it is the removal of rural temperature records (globally) and the the use of “estimated” temperatures that is driving the reports of record global temperatures.
If somone has already addressed his questions please point me to the link.
Hi rogue,
As far as Steven Goddard goes, he has a tendency to make things up. In the Berkeley dataset, at least, we have more stations than ever in recent years: http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/global-land-TAVG-Counts.pdf
(The dip at the end is because not all stations report instantly; some countries still use paper records!). No rural temperature records have been “removed”.
The old GHCN-Monthly dataset does have fewer stations in recent years, but not because rural stations are “dropped”. Rather its because they did a big data collection effort in 1992 to get prior records from all over the world, and set up an automated reporting system post-1992 that used a smaller network of well-distributed stations. However, the old GHCN-Monthly product is going away shortly, to be replaced with GHCN-Daily, which has no drop off in stations in recent years similar to the Berkeley Earth compilation. In either case, the resulting temperature reconstructions are largely identical.
“More stations than ever” you say Zeke, as if that’s somehow a good thing. But we know at least one, Rutherglen in Oz, is a truncated nonsense station only starting in 1965. Isn’t it possible BEST contains “more stations than ever” like Rutherglen? The only answer is yes. It’s possible. I’d say probable.
Since I do not read articles that are so long, I need to ask these embarrassing question:
What did our authors say about Puerto Casado, which was so important to Booker?
Look at http://www.notalotof peopleknowthat.wordpress.com
You can of course check the station record.
http://berkeleyearth.lbl.gov/stations/157455
What do you know. Station moves that coincides perfectly with breakpoints. Of course it should be adjusted.
Homewood did of course not mention that.
Rooter
>”Homewood did of course not mention that.”
Homewood did of course not mention that because he was referring to GISS/GHCN, not BEST.
The BEST site move adjustments are 1971 (about 0.8) and 2006 (about 0.6) ish, total cumulative adjustment say 1.4. Only 2 adjustments made.
http://berkeleyearth.lbl.gov/stations/157455
GISS/GHCN raw and final:
http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_308860860004_1_0/station.txt
http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_308860860000_14_0/station.txt
GISS/GHCN 1951 total cumulative adjustment -1.7, about 0.3 greater than BEST
1967 cum adj -1.78, 1972 cum adj is 1.0 (missing data at 1971). 1971 site move step is therefore 0.78, same as BEST.
Except if you look at the raw data in the 5 yrs immediately prior to 1971, the trajectory of the data was on the way down and actually matches the data on the other side of 1971 at 1971. So why was a -0.8 adjustment required by both GISS/GHCN and BEST when the data was little different immediately either side of 1971?
There was a bigger break in the opposite direction at about 1987 that GISS/GHCN adjust for but BEST does not (see # below). And another at 2004 that neither GISS/GHCN nor BEST adjust for.
1980 cum adj is -0.3, about 0.3 less than BEST
1985 cum adj is +0.11, opposite to BEST (# see above)
1990 cum adj is -1.15, about 0.5 more than BEST
1995 cum adj is -0.65, similar to BEST
2004 cum adj is -0.01, about 0.6 less than BEST
2006 cum adj is -0.01, about 0.6 less than BEST
Missing data precludes nominal 5 yr intervals.
GISS/GHCN do not make the -0.6 2006 site move adj that BEST makes. But they obviously make several more adjustments than BEST do.
Now do you see the problems here Rooter?
After a longer introduction richardcfromnz says:
“GISS/GHCN do not make the -0.6 2006 site move adj that BEST makes. But they obviously make several more adjustments than BEST do.
Now do you see the problems here Rooter?”
The two adjustment algorithms does not end up with the same result.
Of course the don’t. How could they? Different algorithms in use and there are also different number of stations the algorithms can utilize. Of course that will lead to differences on station level. It is a problem that you don’t understand that.
Rooter
>”The two adjustment algorithms does not end up with the same result. Of course the don’t. How could they? Different algorithms in use and there are also different number of stations the algorithms can utilize. Of course that will lead to differences on station level. It is a problem that you don’t understand that.”
Oops, “differences at station level” from different algorithms?
Speaks volumes Rooter.
I notice you side-step this question:
“So why was a -0.8 adjustment required by both GISS/GHCN and BEST when the data was little different immediately either side of 1971? ”
And you don’t address the fact that there was a bigger break in the opposite direction at 1987 that BEST doesn’t adjust for but GISS does.
Was this a bit too tricky for you Rooter?
Steven Mosher | February 10, 2015 at 11:58 am | Reply
Note that I linked to one of the others who made an issue of BEST.
Booker raised the issue WRT NOAA, OTHERS ( see the link ) made the issue BEST as well.
read harder; comment less
Booker article does NOT refer to “NOAA, OTHERS” It is based on Paul Homewood’s website, which has never referred to BEST or has any analysis of BEST data.
Maybe read more accurately…………..
If BEST is ok and the others surface indexes matches BEST.. What is the problem for you?
Booker’s claim based on Paul Homewood’s blog is just crap. They see that there are different values in gistemp 1987 met stations index compared tho the gistemp land-ocean index in 2015. Even excluding the oceans the met stations quadrupled from 1987 to 2015. The better question would be why were those indexes not more different?
You just don’t get it do you, or you haven’t actually read all of Paul Homewood’s latest posts.
Historic data in Iceland proves the original temperatures were correct, they have been massively adjusted by the inability of a simple Computer Algorithm to recognise reality from bad values.
Of course Mosher & co say but it is only Iceland, but “only Iceland” catagorically proves that the Algorithm does not work properly because of the Assumptions built in to it.
Just like GCMs don’t work because of the Assumptions built in to them as well.
Osborn: Is that all you have? The scandal is wrong adjustments in Iceland? Well, take comfort. There are probably more wrong adjustments in the Arctic in GHCN (not necessarily BEST though). Taken together these adjustments in the Arctic reduce the Arctic temperature trend in this century.
A scandal? But not a warming adjustment like Homewood tried to convey.
In delingpole see the links
See posts on watts. That link got dropped.
I can add it if you demand
The compilation and analysis of temperature data from worldwide stations as a measure of global surface temperature is a natural thing to do. What the results mean is a completely different story. While it’s a temperature record of some kind, it’s not at all clear to me that it is fit for purpose as currently used. Just my opinion.
The fact that Robert, Zeke and Steve have accepted the chore of QC’ing the data and it’s manipulation and are working to convince the rest of us that this process is transparent and unbiased is laudable. Especially given that they seem to be doing this on a voluntary basis. Cudos guys and thanks for taking the abuse. Been there, done that.
However at the end of the day, and accepting the BE results at face value, we are talking about a change of less than 1 deg. C in a hundred years, with interannual fluctuations of tenths or hundredths of a degree. This is interesting but not particularly meaningful from a scientific or climate perspective.
We know that atmospheric CO2 concentration has also increased during the past hundred years and based on “green house science” we presume that some part of the temperature increase is resulting from increased CO2. Personally I don’t think the CO2 contribution is all that much.
None of this is very alarming to me nor should it be to reasonable people. The alarm arises only after the introduction of the GCM’s and their systematic misuse to project future high levels of warming that may or may not be cause for alarm. The fact that model projections are not consistent with measured warming during the past 15-20 years is a pretty good reason to question the advisability of making extraordinary policy decisions on the basis of the models.
I think the terms of fraud and hoax are more applicable to misuse of the temperature data re. warmest year diatribes, misrepresentations of the veracity of the models and misleading assertions of extreme weather events being a consequence of CO2 driven climate change.
Pingback: Rising sea level, melting ice or...? - Page 5 - Fuel Economy, Hypermiling, EcoModding News and Forum - EcoModder.com
So, wait, BEST still includes stations like these?
http://www.surfacestations.org/ca/hansen50.jpg
Of course it does. A great many Australian BOM temperature stations have the Stevenson screen sitting on top of a heavy STEEL frame…. nice heat sink there, you cannot touch them on a hot day! Just look at what the BOM thinks is “good exposure”…. steel frame, next to a steel fence: http://www.bom.gov.au/climate/cdo/about/sites.shtml
Lucifer: You can of course leave it out. But by what criteria? How to decide?
The main problem for you here is that you do not like that the adjustments are spot on in this case. The adjusted data represent the temperature history far better than the raw unadjusted data.
BOM Oz ACORN adjustments.
http://www.warwickhughes.com/blog/?cat=48
rooter,
You wrote –
“Mike Flynn really got problems when discovered that the BEST algorithm adjusted the station baddy in Tucson. Everything was suddenly chaotic. Looks like a breakdown of some sort.”
I assume that English is not your native language, based on your written expression. This might explain your apparent inability to comprehend what I wrote. Might I tactfully suggest you reread what I wrote. Let me know if you don’t understand, and I will put those parts more simply.
I admit I cannot understand what you wrote. What is a “station baddy”? Why would it suddenly become chaotic? Why would it break down?
Is this an example of some new Warmist language, or just more bizarre Warmist obfuscation?
I realise that expecting Warmists to provide rational answers to simple questions is probably an expectation too far, but one can always hope!
Live well and prosper,
Mike Flynn.
Mike Flynn does not understand why a station like this:
http://www.surfacestations.org/ca/hansen50.jpg
is a station baddy. Does not understand why its temperature record is biased. And why it is a good idea to adjust the series.
I believe it is this:
http://berkeleyearth.lbl.gov/stations/27670
Sure are much to adjust there. Lots of upward breakpoints. That means homogenizing results in clearly reduced trend.
But as I understand, that is not allowed. No adjustments!
Anyhow. In the file from BEST there are unadjusted and adjusted. Red unadjusted. Blue adjusted.
http://i.imgur.com/RnlHepa.png
A scandal!
You’re focused on the trends.
I’m focused on the fact that the station is worthless.
rooter,
If the components of the Earth system behave chaotically, pointing at one station, a group of stations, or all stations, and endeavouring to draw a useful conclusion might be completely pointless.
Have you any reason for believing that following this trend is not merely placing you closer to an inevitable inflection point?
Can you produce any evidence that the three dimensional movements of the atmosphere, aquasphere, and lithosphere are not unpredictable in any meaningful sense?
Is there a point to this endless examination of the estimated past?
Do you really believe that the past extrapolates to the future? If you do, most casinos will provide you with data showing individual results of roulette wheel turns. They love people who believe that the past predicts the future.
It’s all good for a bit of light amusement and relaxation – albeit at great expense to someone – not much good for anything else, what?
Live well and prosper,
Mike Flynn.
Mike Flynn really got problems when discovered that the BEST algorithm adjusted the station baddy in Tucson. Everything was suddenly chaotic. Looks like a breakdown of some sort.
rooter,
You wrote –
“Mike Flynn really got problems when discovered that the BEST algorithm adjusted the station baddy in Tucson. Everything was suddenly chaotic. Looks like a breakdown of some sort.”
I assume that English is not your native language, based on your written expression. This might explain your apparent inability to comprehend what I wrote. Might I tactfully suggest you reread what I wrote. Let me know if you don’t understand, and I will put those parts more simply.
I admit I cannot understand what you wrote. What is a “station baddy”? Why would it suddenly become chaotic? Why would it break down?
Is this an example of some new Warmist language, or just more bizarre Warmist obfuscation?
I realise that expecting Warmists to provide rational answers to simple questions is probably an expectation too far, but one can always hope!
Live well and prosper,
Mike Flynn.
Global temperature: a towering and intricate mathematical structure wobbling on its base of old tripe.
Really, all that is left to do is to compare the “raw data” used in BEST and other analyses to the actual paper records generated at the time (the true raw data). If someone like Mosher assures me they are exactly the same, I am quite willing to take his word for it as I have no reason to doubt him, and plenty of reason to trust him.
I have a nice Bridge in London for sale, would you like to buy it?
1.6 billion records.
I’m on it.
But you realize that the brandons of the world would remain skeptical. You know there is uncertainty over exactly how many died in the holocaust.
Steven Mosher,
Am I correct in thinking that the supposedly raw data you analysed is really data you made up, because there were too many real records to examine? Or did you just look at a handful of original records, and get bored?
You wrote –
“1.6 billion records.
I’m on it.”
No you’re not. You’re making that up too.
Guessing, estimating and modelling, pretending that statistical sampling is equivalent to rigorous scientific examination of recorded results, can produce nonsense. In this case it has. Warmist PC nonsense, to be sure, but nonsense nevertheless.
Get cracking. Examine 1.6 billion records, if such there are. It looks like a lifetime job, at the very least. Good luck to you! I can’t see any point at all, so I won’t volunteer to help. Maybe you could recruit some of the millions of Warmists to assist in such an important task. I’m sure they’d come rushing to your assistance.
Or maybe not.
Live well and prosper,
Mike Flynn.
Really? you write “You know there is uncertainty over exactly how many died in the holocaust.” Repulsive.
Mosher used a lower case ‘h’ and had some numbers in his stuff. Try that, Brandon.
The data in Bolivia and Paraguay has definitely been corrupted by GISS ect My father was responsible in 1964 to 1977 to fix all the stevenson boxes here for the UN WMO. The RAW data from these areas is correct and it has been fraudelently corrupted beyond belief. He would be revolving in his grave if he knew how corrupted the WMO ect has become. He is one of the persons mentioned here just to justify LOL
http://www.rti.org/pubs/bk-0003-1109-chapter04.pdf
Why the need for fixing the Stevenson shelters if they were ok?
Perhaps you inadvertently told the reason for the breakpoints in Paraguay 1960-1980. And the reason for the adjustments in Paraguay
Stevenson shelters need regular maintenance. The whitewash wears off and has to be redone for example. The fact that someone was employed to maintain the instruments is an assurance that the data is of good quality, not a sign that it should be distrusted.
If data from instruments I was responsible for was adjusted in this cavalier fashion I know I would be annoyed. Such adjustments are a slight on the scientific reputations of those who gathered the data. It is tantamount to a suggestion that they were so incompetent they couldn’t use a thermometer properly.
Let me add to my reply because it is worse than that.
Suppose you are correct and maintaining the Stevenson screens did lead to the breakpoints you allude to. A Stevenson screen which is bare of paint or lacking in ventilation or otherwise in bad condition usually records a spuriously high temperature. Such issues develop slowly as the instrument deteriorates. Fixing the problem on the other hand causes a sudden correction, a temperature drop which will be detected as a breakpoint. So what should we do about sucha record?
What breakpoint correction procedures do is shift the two parts of the temperature record to align them and eliminate the ‘break’. But remember that the break in this case is actually the correction caused by fixing the instrument. The bias caused by the slow deterioration of the instrument is not detected as a breakpoint and the warming that this causes is preserved.
Breakpoint correction in this situation is a systematic procedure for undoing all the good caused by actually maintaining the instruments.
Ian H:
Again: Why the need to fix the screens if they were ok in the first place?
Your logic that microsite changes that results in a cooling bias should not be taken into account is very strange. You could have answered your own question of what to do with such records. No adjustments for changes?
Do you have any reason to suspect that the distortion to the record caused by a pattern of instrument deterioration and periodic maintenance causes a spurious warming or cooling trend? I certainly can’t see the justification for believing this. If not do nothing.
So yes – no adjustments at all for changes.
Ian H says he will not adjust for shelter changes.
Nice to know.
I am still concerned about the 2012 presentation by E. Steirou and D. Koutsoyiannis to the European Geosciences Union. It is only available as a power point. The analyzed the effects of homogeneity adjustments on 181 GHCN stations and found that half of the warming trends were a result of the adjustments, a far cry from what we see from Berkeley Earth.
Now their sample was not geographically very representative, but that number of stations usually can reproduce global (if evenly distributed) or regional changes quite well. Further, I don’t have a problem that correction for time of day issues–which is somewhat implied in their methodology–could induce a scientifically defensible warming.
The second author is quite highly regarded in the field of statistical hydrology, and yet the authors have been in radio silence about this result; it hasn’t turned up anywhere in the literature (with props to the fact that this is a very contentious field, and editors are in fact intimidated by some of my more goon-like friends) and you would think if it were robust and convincing it would have appeared somewhere. An additional reproducibility problem is that it is the dissertation of the first author and that is in Greek.
Any thoughts on this one with regard to Berkeley Earth?
Here are the slides that Pat Michaels refers to:
http://88.167.97.19/temp/Investigation%20of%20methods%20for%20hydroclimatic%20data%20homogenization_2012EGU_homogenization_1.pdf
There was a lot of discussion when this first came out, some of which is listed and linked here: http://www.itia.ntua.gr/en/docinfo/1212/ including another link to the presentation. All before BEST, so a good question how BEST relates?
“From the global database GHCN-Monthly Version 2, we examine all stations containing both raw and adjusted data that satisfy certain criteria of continuity and distribution over the globe. In the United States of America, because of the large number of available stations, stations were chosen after a suitable sampling. In total we analyzed 181 stations globally. For these stations we calculated the differences between the adjusted and non-adjusted linear 100-year trends. It was found that in the two thirds of the cases, the homogenization procedure increased the positive or decreased the negative temperature trends.
Problems.
1. GHCN V2. this is DEPRECATED. BAD Skeptic.
2. Continuous records. You have to be careful with GHCN -M because they can combine records from multiple sources even when those stations are different.
3. No code
4. No data.
waste of my time.
A large part of those stations were for the continental US. And we know the TOBS issues in the US. No wonder they got a large proportion of adjustements. No wonder that presentation did not result in a paper either.
Rooter, incorrect. Europe is well represented. Asia is well represented. Oceana is well represented. Only South America and Afrika are underrepresented. Page 9 of 18.
Mosher, the sample is 181 GHCN stations, essentially all that had a reasonably continuous 100 year record. Warming bias in NCDC homogenization.
Rud Istvan do not quite see what he writes:
“Rooter, incorrect. Europe is well represented. Asia is well represented. Oceana is well represented. Only South America and Afrika are underrepresented. Page 9 of 18.
Mosher, the sample is 181 GHCN stations, essentially all that had a reasonably continuous 100 year record. Warming bias in NCDC homogenization.”
54 of 181 from Norht America. That is not a large proportion.
Wellwell.
And because they are using only long series, what does that tell us about adjustments after for example 1970? Not much.
Too few stations
Er., I thought you could knock out the majority of stations and still get the same lineshape. I recall you stating that the CUS could be done rather well with only 100 stations.
For evaluation of homogeneity approaches.
Yet another lazy response from SM, a pompous grunt as it were. There are no minimums in sampling theory, so your grunt is meaningless.
It is possible with 182 well-sited stations to get a reasonable mean. But it’s not much use for sub-sampling. And papers like Lawrimore et al 2011 have analysed the whole collection, to see the actual effect on a global index. So have BEST, as shown here. Even I have. Why would a journal be interested to publish such a limited set? Well, I suppose it beats Booker’s.
One cannot but note that all of the temperature reconstructions have essentially the same shape and apparent delta Temp for all pairwise comparisons; thus they are all either correct or all fatally flawed.
I also seem to recall that the smallest magic number for Jack-knifing is a true n=24, which gives triplicates of n=8, the magic n=number.
Delta T is important as it gives us a way to arrive at a TCS, which on the bottom line is what we are all interested in.
Hi Pat,
Regarding Steirou and Koutsoyiannis, the lack of any station list prevents replication. Its fairly trivial to just do a global temperature reconstruction with and without homogenization (as we’ve done here, and other folks have done elsewhere) and see that the global effect is relatively small. There are certainly regions and individual stations where homogenization has a much larger impact, and it may well be that Steirou and Koutsoyiannis happened to select a subset of stations where that was the case. This does not mean that the adjustments they found are unjustified, as there are real systemic biases (TOBs change, instrument changes, etc.) that introduce non-random errors into the raw data.
I think the most compelling evidence that adjustments are not creating additional bias come from tests using synthetic data, e.g. the Williams et al 2012 paper: ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/williams-etal2012.pdf
We also presented a poster at the AGU a few years back expanding this approach to the Berkeley data: http://static.berkeleyearth.org/posters/agu-2012-poster.png
Well, no, Zeke. The presentation is a summary of a phD thesis in Greek. That apparently contains what you would seek. Hint: both authors are also fluent in English. A global requirement these days. Have you asked?
Zeke,
Thank you for your helpful response. At the risk of trying your patience can you provide any insights as to the claimed divergence between BEST and satellite records that Steve Goddard cliams in this chart?
https://stevengoddard.files.wordpress.com/2015/02/screenhunter_6940-feb-08-11-50.gif
regards
Rogue
There are simpler plots. WoodforTrees can do a monthly comparison plot for BEST v. RSS. Growing anomaly divergence since 2000, now on the order of BEST 0.25C warmer. It would be interesting to get an explanation
There is a pretty substantial disparity between land and satellite records over Alaska and Northern Canada (and I would guess, also Siberia, etc….). I noticed it when working over some Alaska data, and what came to mind is that the satellites can’t see down to the level of the winter inversions. Given that cold air should preferentially warm (because the wavelengths where there is water/carbon dioxide overlap are more saturated in a wetter (warmer) atmosphere, so the effect of co2 is not very logarithmically damped), that would mean that the satellite would not be picking up the very low-level warming and hence could be a source–not the only one–of the disparity. I ran this by Christy when I was looking at this in 2011 and he agreed.
I agree with this also.
But aren’t cases where warm air overrides cold more common?
TY Pat and Judy. Makes sense. Testing current measurement systems against each other for consistency and ‘modern’ quality is something I have been exploring recently for the period since 1998, and especially since ~2005 and ARGO. Just consistency, not trends.
WFT, last I looked, had old data. sent him mail. no response.
RSS has a problem with NOAA15 according to Roy.
Now I suppose you guys want me to defend RSS adjustments.
do I get a cookie at some fricking point or just more allegations of being part of a fraud?
Climate Explorer has the RSS lower troposphere product in degrees K which is kind of nice. That indicates the average altitude they are measuring is about 2.5 km at around 271 K degrees.
http://climexp.knmi.nl/data/irss_tlt_0-360E_-90-90N_n.png
Cookie! No cookie! Just ‘thanks from a grateful nation…’
Well, I prefer moshe raw to moshe adjusted.
================
Pat Michaels:
” Given that cold air should preferentially warm (because the wavelengths where there is water/carbon dioxide overlap are more saturated in a wetter (warmer) atmosphere, so the effect of co2 is not very logarithmically damped)”
I’ve seen this before but I’m not convinced it’s correct.
Take the extreme cold air T=0K, by definition no energy emitted, and so, zero radiative forcing from increasing CO2.
Now, radiative forcing depends on the profile of temperature, so things vary with height. And the net flux from 2xCO2 going into the surface is greater at the poles and lesser (even negative!) at the equator, suggestive of the overlap effect that you raise.
But the radiative forcing aloft is lower at the poles than the equator which we see in the upper left plots here:
https://scienceofdoom.files.wordpress.com/2013/03/hansen-1997-plate2-forcings-temperature-2xco2-annotated.png
So, rather than cold air should preferentially warm I’m thinking that the surface beneath inversions ( which, yes, are cold ) should preferentially warm, but that it is actually the warm air (tropics) that should also preferentially warm due to increased CO2.
One other aspect of polar radiative forcing comes from Dr. Curry:
curry.eas.gatech.edu/currydoc/Curry_JAS40.pdf
Polar air masses are very dry, but also loaded with the ubiquitous ice crystals which are ( I’m guessing ) pretty good IR emitters and also ( I’m guessing ) probably poorly included in models.
RSS and UHA both adjust their “data” its actually not temperature, Read their documents. find out how and why they adjust data for
1. Changing sensors ( different satillites)
2. Changing time of observation ( overflight paths migrate I believe.
3. Changing location ( orbital decay)
YUP the same stuff we have to correct for they have to correct for.
Ask for their code and data. Then give me a cookie and I’ll look at it.
Then ask how can I compare a Tmax/ Tmin estimate with a
satellite estimate..
especially if AGW expresses itself in a change to Tmin?
Steven, ignoring your immediate previous tantrum, I have read everything available from both UAH and RSS on how they interpret the MSU results at the various frequency channels. And also BEST, GISS v3, NCDC PHA, and even NOAA nClimDiv. Piecing together a temp measurement puzzle.
As for the satellites, bird shifts are easy. Step changes, easy to calibrate from overlapping time periods. They do. No different than sat altimetry for SLR (essay Pseudo Precision). TOBS equivalent is equatorial drift time in polar orbits. Known, easy to compensate, see essay in new book. Excuse was imcorrectly used for a massive ‘FAIL’ paper on UTrH constancy called out in the climate chapter of the my book (has importantly to do with WV feedback). Orbital decay (altitude) not so important. The hard one is instrument degradation over time in space. Unshielded radiation damages electronics over time. The differences between UAH and RSS mainly apparently involve this last on NOAA15. Or so they both seem to think.
No cookies for you, sorry. I do my own work on all this.
“especially if AGW expresses itself in a change to Tmin”
LMAO! That is just pure guff. We can’t show warming, so we say that no, it’s the rising Tmins. You guys are so funny!
Well, you done the BEST that you can.
And it remains consistent with warming at the low end:
MODEL: IPCC5 (RCP8.5): 4.2C/century
MODEL: IPCC4 Warming High: 4.0C/century
MODEL: Hansen A: 3.2C/century ( since 1979 )
MODEL: Hansen B: 2.8C/century ( since 1979 )
MODEL: IPCC4 next few decades: 2.0C/century
MODEL: IPCC5 (RCP4.5): ~2.0C/century
MODEL: Hansen C: 1.9C/century ( since 1979 )
MODEL: IPCC4 Warming Low: 1.8C/century
———————————————————————
Observed: NASA GISS: ~1.6C/century ( since 1979 )
Observed: NCDC: ~1.5C/century ( since 1979 )
Observed: UAH MSU LT: ~1.4C/century (since 1979 )
Observed: RSS MSU LT: ~1.3C/century (since 1979 )
MODEL: IPCC5 (RCP2.6): 1.0C/century
Observed: NCDC SST: ~1.0C/century ( since 1979 )
Observed: RSS MSU MT: ~0.8C/century (since 1979 )
Observed: UAH MSU MT: ~0.5C/century (since 1979 )
———————————————————————
No Change: 0.0C/century
Data from mercury in glass, bimetal, or thermocouple instruments isn’t temperature either. It’s a measure of volume expansion of mercury, the differential expansion of two different metals, or the voltage differential between two different metals. No temp is measured in any of the three cases. So the microwave method is no different in that respect. You are just trying to muddy the conceptual waters.
Robert Rohde, Zeke Hausfather, Steve Mosher:
Thank you for another good presentation.
Thanks matthew
Robert Rohde, Zeke Hausfather, Steve Mosher,
There seems to be an infestation of duplicated data in your analysis. I found the same with GHCNM and notified them about it. Whether they will fix it at some point is still to be determined. They have more than 450 pairs of excessive data duplication at two stations in a single year. 184 of those are all 12 months. Some of the station pairs duplicate data for a decade or more. By duplicate I mean stations within a single country having 7 or more months of identical data within a single year. This should be rarer than a massive asteroid strike. Most of these occur during the overlapping GHCN-GISS anomaly periods 1951-1990 with no excessive duplication this century. It seems people had a strange penchant for tidying records containing ugly missing data months especially during the anomaly period.
For comparison I checked the USHCN(4250*) subset of GHCN using +- 0.05C to eliminate the 100ths decimal since most other stations don’t report that figure, and rounding methods used may not be consistent. That subset has zero occurrences of more than 7 months duplicated and only 4 of those out of more than 78 million comparisons.
Took a quick look at your data files and immediately found these two station pairs which demonstrate what I am saying. I don’t intend to do a comprehensive analysis of your dataset. What I can also say is, the ISTI database which is about the same size as your’s is roughly 10% duplicate data.
Berkeley ID#: 151423, Berkeley ID#: 151425 1951-1960
Berkeley ID#: 153938, Berkeley ID#: 158939 1961-1971
I don’t know how much difference it makes globally, but I suspect it must have an effect on calculating breakpoints and result in erroneous weighting being given to those monthly values regionally.
see my comments above.
Unless the data from stations ( there locations, names, and data )
is duplicated for the entire record, they are kept as separate stations.
That means if the location, name and data dont match exactly they
are kept as separate stations.
The other option is to merge stations that are slightly different and “assert’ they are the same.
If you check the status updates for GHCN Daily ( for example ) one of our primary sources, you’ll see that there are a pretty good stream of changes where “false duplicates” get removed.
false duplicates make your estimate more certain than it should be.
Steven Mosher,
I notice you didn’t mention anything about the station pairs I listed. You also didn’t dispute it could have an effect on breakpoint calculations which I believe you calculate by looking at surround stations. Each station in those pairs above would lead you to assume no breakpoint in the other during the duplicated portion of the time-frame. You also didn’t dispute the idea of those monthly values getting more weight in a regional analysis.
I don’t know about your dataset other than the cases I mention above. I must say it sure looks odd to find most of the duplication in GHCN occurring during the anomaly periods. Is it that way in your dataset?
If GHCN-D is concerned about duplication, don’t you think checking your own dataset to uncover and remove such data would be wise?
Maybe you should run a check by deleting all such duplicate data prior to doing an analysis. At least then you would have an idea how much of a difference it makes. I can’t comprehend a reason for leaving erroneous data in the database. It’s not like you have a shortage of stations.
I looked at the first pair.
different name
different identifier
different location
different data
But your science is settled. they are the same?
Now imagine I had joined them
some guy would argue the opposite case.
Here is what I also know. pick 1000 stations.
answer doesnt change
Are you asserting that http://berkeleyearth.lbl.gov/stations/151423
Is the same as http://berkeleyearth.lbl.gov/stations/151425
We have two Stations because the location is different
The name is different
And the data is
Different
Steven,
You seem to think it is OK to have dozens of records from single location as long as those records have been assigned slightly different locations or names and some years of data are different values thereby making them all independent sources for the entire length of their record. This gives each of them equal weight with other truly independent records which would only have a single input into the analysis. Sounds like consensus building.
The idea that most excessive duplications are found in the anomaly periods rather than in the other portions of the record strongly suggests finagling has been done. All that duplicated data should be removed. It’s not like their use is required. Plenty of other stations available.
More than 93% of the within country two station comparisons in the entire GHCNM dataset had zero matching months of data in the same year. That applies for the subset of stations strictly within the USHCN and also for the remaining portion of the world’s stations. The rest of the world is where most of the duplication has happened. Do the calculation for the likelihood of finding any 12 months matching in a single year between two stations in the same country. You are looking at trillions to one against it happening.
Bob the wmo numbers are different as well.
There appears to be a partial overlap. But we don’t assert that stations are identical unless they are substantially the same in position names identifiers and data.
Sounds like you are OK with leaving obviously erroneous data in the database and then using it.
Look at the plots.
1. the location is different.
2. the name is different
3. the identifier is different.
4. the data is different.
You to argue that both of those records came from the same place?
go ahead.
here is another hint. Sometimes stations have two sensors. and the records go to different agencies.
Mosher, The Australian has a front-page story on the issue today. It concludes:
“However, a report on US climate scientist Judith Curry’s website yesterday rejected any claims of fraud in the homogenisation process. The report by Robert Rohde, Zeke Hausfather and Steve Mosher said it was possible to find stations that homogenisation had warmed and others that had cooled. It was also possible to find select entire continents that had warmed and others where the opposite was the case. “Globally, however, the effect of adjustments is minor. It’s minor because on average the biases that require adjustments mostly cancel each other out,” the report said.
“In a statement to The Australian, NOAA said it was understandable there was a lot of interest in the homogenisation changes. “Numerous peer-¬reviewed studies continue to find that NOAA’s temperature record is ¬reliable,” NOAA spokesman Brady Phillips said. “To ensure accuracy of the -record, scientists use peer-¬reviewed methods called homo¬g¬enisation to adjust temperature readings to account for a variety of non-¬climate related effects such as changes in station location, changes in observation methods, changes in instrumentation such as thermometers, and the growth of urban heat islands that occur through time,” he said.
“Mr Phillips said such changes in observing systems cause false shifts in temperature readings. “Paraguay is one example of where these false shifts artificially lower the true station temperature trend,” he said. Mr Phillips said the largest ¬adjustment in the global surface temperature record occurs over the oceans. “Adjustments to ¬account for the transition in sea surface temperature observing methods actually lowers global temperature trends,” he said.”
http://www.theaustralian.com.au/national-affairs/climate/arctic-monkeys-climate-agencies-revise-weather-records/story-e6frg6xf-1227215290929 (paywall)
Pingback: The Climate Change Debate Thread - Page 4646
Pingback: Global Warming - The "settled" science unwinds. - Page 19
“And I used to be very suspicious… I think that Booker and Delingpole suddenly jumping all over this will eventually reflect badly on them…”
No long skeptical? The entire land record is fatally flawed by collecting raw data in the midst of urban heat islands. “Up to 80 per cent or more of the Earth’s surface,” reports Christopher Booker (…STILL being tricked with flawed data on global warming), “is not reliably covered.” Fewer than 6,000 official thermometers (with their individual readings being extrapolated to cover up to 1.6 million square miles) are used to arrive at the average temperature of the world; and, since about 1900 most of the official thermometers “are concentrated in urban areas or places where studies have shown that, thanks to the urban heat island effect, readings can be up to 2 degrees higher than in those rural areas.”
I think any good scientist ought to be a skeptic. ~Freeman Dyson says,
Waggy,
All I said was that BEST has done pretty much as well as anyone can with the available data, and that Delingpole and Booker are mistaken if they think they can stop alarmism in its tracks by asserting fraud in adjusted data.
Oddly, Mosher jumps down my throat every time I mention how I became a sceptic but lets it pass when I say that I used to be suspicious of the temperature adjustments, but he and Zeke (OK, mainly Zeke) have convinced me that the BEST approach is more or less sound.
Anyway, I agree completely that the coverage and quality of the data is totally inadequate to the task of measuring any sort of average global temperature (absolute or anomaly) to within 0.1°C. And in any case, as noted elsewhere, it’s moist enthalpy we should be measuring anyway, not just temperature.
Am I still sceptical? Sure. I’d call myself a microwarmer (as in AGW is real but too small to worry about and probably too small to measure with the network we have now). But shouting about temperature record adjustments is the wrong way to go to fight the politicised alarmism of Greenpeace et al.
In the first link Christopher Booker reports, “One of the more provocative points arising from the debate over those claims that 2014 was ‘the hottest year evah’ came from the Canadian academic Dr. Timothy Ball when, in a recent post on WUWT, he used the evidence of ice-core data to argue that the Earth’s recent temperatures rank in the lowest 3 per cent of all those recorded since the end of the last ice age, 10,000 years ago.” If true it invalidates the whole approach to gathering data than can be massaged to sell a story temperatures are hotter now than ever.
Wag, the ice core data Ball referred to is mostly GISP2 from Greenland. He was probably incorrect to stretch that location to the world. However, if one uses the Marcott thesis–not the abominable science paper— then global paleoproxies indicate that perhaps half of the past 2000 years was globally warmer than now.
Good points and we know the paleoproxies have been abused to make a case for warming that just does not exist, a more recent example being Mann’s ‘hockey stick’ science. “Overall,” says Wegman when testifying before congress, “our committee believes that Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.”
My take is that none of it matters because sea levels and overall temps have done nothing but swing for millennia and it’s hard to find anything odd in the utterly commonplace.
But, regarding UHI, it’s interesting to reflect that most of Sydney’s (Obs) hottest years are indeed clustered after 2000, while two long-record coastal+rural stations I just happened to glance at this morning for comparison have their top temps clustered well back in the past.
Yamba Pilot Station’s 95th percentile of hottest years were between 1884 and 1896. Kempsey, opened later and further from the coast, had its entire 95th percentile in a straight run from 1910 to 1919 (and that’s within BoM’s safer screening era). Similar story for the 90th percentile at these stations. Other and more mixed stories from elsewhere, of course.
A history of max temps tends also to be an history of cloud behaviour, land clearing, re-vegetation, screens, UHI etc so it’s hard to care much. (Contrary to what has been cleverly half-implied, much of Australia was drier for the half-century before 1950.) If people want to turn this vague stuff into graphs and if those graphs indicate some global cooling or pausing or warming in various lines or cycles…well, it’s good to know the world is still the world! We still haven’t flatlined after all these millennia! Still got it!
Of course, a degree of heat in Sydney is far more distinguished than a degree of heat here in the boonies. If a degree falls in the forest…
Mosomoso
I suspect that the rural stations you quoted did not have Stevenson screens and therefore their data will be discounted byBOM who are removing records set pre Stevenson.
Tonyb
tonyb, not always the case in regard to Stevenson screens. Australians were early adopters and BoM often seems not to know (or won’t say) if screens were in place.
It’s also interesting that these heat clusters in the past were preceded and immediately followed by more average runs of temps.
Certainly, with screens in place and everything good to go, that cluster of av max heat post 2000 is not apparent up the coast here as it is in Sydney.
Greetings from the mid coast, where the almost-El Nino has forgotten the rules (again!) and brought us a coolish and damp late summer after a hot start.
Mosomoso
These clusters recur continually throughout the record. We see clusters of heavy rain, then drought, then heat then cold with, as you say, ‘normal’ conditions in between. Which beggars the question as to whether our usual weather state is for extremes or normality. In the uk I would go for extremes
Tonyb
Sydney’s a great call, mosomoso. It’s had a continually published local newspaper since 1860 or so and climate scientists can’t disappear what actually happened, hot days, cold days, drought etc since that time. BOM is left just arbitrarily ignoring history and stating that they only care about temps from 1910, as if the country didn’t exist beforehand. Absurd. Late 19th was real hot on the eastern seaboard in Oz. It just was. It was all over the newspapers. Trust the contemporaneous records. It took the Oz climate scientists Karoly and Gergis too (I think) until 2014 to bother to look at news records to map actual temps from back then and lo, they found it was hot, probably hotter even than today. No great press release for that finding though.
Robert, Steve, Zeke and Judith,
Thank you for this posting. Between this and the ongoing dissection of the F & M paper at Climate Audit I feel like I’m witnessing some of the best manifestations of the new paradigm of Internet science discussions. Even the angry ego clashes, the nattering nabobs of negativism and nitpickers are helpful in comprehending the sausage-making aspects of the evolution of knowledge. Kudos to all. I feel a little smarter today.
Both these discussions seem more like angels on pinheads.
The surface temperature is not remotely the energy content of the atmosphere. The energy content at the surface has both latent and sensible components. These change with surface water availability. Temperature at the surface changes with rainfall and drought and not just ‘global warming’.
The L&M paper purports to analyse for divergence of models from observations. For each of these models there are thousands of feasible solutions. Divergence is a matter of choosing a warm solution from amongst the many. It has no deeper significance. Climate models at best define a probability space – there are no unique, deterministic solutions.
Rohde, Hausfather and Mosher
Your first paragraph states, “— and Anthony Watts previously insinuated that adjustments are somehow criminal.”
I went to your link and did not see where Watts insinuated anything about your BEST efforts. The earlier discussion at that link was all about NOAA adjustments.
a link to the latest cartoon got dropped.
Essententially you had Booker raising an issue.
Delingpole piled on with links to Zeke and Shub who had a swipe at us.
Then WUWT with BS and Josh taking a swipe at us.
Then Morano and drudge reposting an old article where the claim of criminal activity was made.
So the minute all of those guys want to say that we are not a part of a hoax or fraud or criminal activity I’ll be satisfied..
Stop showing your thin skin. This is politics, Chicago style. Obama style.
Josh’s cartoon had nothing to do with BEST. Already posted before upthread. The MSM columns you cite have nothing to do with BEST.
Your paranoid (for evidence see reply upthread ending in ‘show thempiles of ashes’), increasingly vehement, and increasingly irrational responses do, however, suggest something might be rotten in BEST’s Denmark.
To paraphrase the Bard.
So, to rehash previous jousts. BEST station 166900 is just a temperature expectation field, not a temperature. BEST station 151882 is just a data/metadata ingestion glitch… But it all evens out. Trust you. Well, learned long ago in Chicago style politics, never trust, only verify.
How about BEST doing that on the central South America and partial Arctic regions Homewood specifically and irrefutably criticized concerning GISS?
Booker and Delingpole do not constitute a ‘pile’. No one took a swipe at BEST, it was used to fact-check GISS’ adjustments. Too bad Berkeley makes the same type of changes to data the other agencies make.
All this sturm und drang over whether the made up, computer generated data was massaged in the optimum way.
People are missing the forest for the twig.
1. Nothing you do with statistics will generate a number that can with any seriousness be called a “global average temperature” with the precision and accuracy claimed by the reporters, let alone that necessary to implement massive global intrusion by the government into the energy economy.
2. Notwithstanding point 1, the CAGW movement desperately needs to claim that they do know the GAT to within tenths of a degree per decade so they can push their decarbonizaton/socialization policies.
3. Actual accuracy and precision are irrelevant for political purposes, what matters is the headlines.
“WARMEST YEAR EVER! (by .002 degrees, with 35% probability)
4. The reported temp trends were full of methodological statistical errors, which undermined the headlines.
5. BEST was created, not to correct the reported trends (because ‘correctness’ is irrelevant and unattainable), but to ‘fix’ the statistical problems and so improve the ‘suitability’ of the reported GAT for headline purposes.
It is hilarious in this context that one argument in support of the supposed objectivity of the BEST adjustments is that in some instances their statistical legerdemain results in ‘cooling.’. It is hilarious because the miniscule adjustments BEST makes in the reported record, you know, the proof that BEST is objective ‘science’, was the intended goal of the BESTers.
“See? We can prove we are objective and did not set out to provide PR support for the CAGW movement because our results are exactly what we predicted them to be. Within hundredths if a degree!”
(And don’t get me started on ‘predicting’ past temperatures.)
BEST may be the best ‘we’, meaning they, can do. But that does not make their work sufficient for the purpose for which it is being sold.
Logic has never been one of Mosher’s strong suits.
I think the most interesting development of every one of these efforts by warmists to defend their work is that they expose just how incredibly shaky the work really is. Somehow, ‘we’re pedaling as fast as we can’ is supposed to prove their claim that they can get their bike to the moon.
Thought fer Today. ( I have the Franchise. )
‘At best we can believe in the general direction of travel of
the local instrumental record – especially when backed by
such things as crop records/ observations – but not on
their accuracy to tenths of a degree.’
H/T. Tony B.
http://wattsupwiththat.com/2011/05/23/little-ice-age-thermometers-%E2%80%93-history-and-reliability-2/
Serfs wonder how changes based on uncertain assumptions
make the data more reliable … rock on rock or more like
turtle on turtle all the way up or down.?
Trust buster. A serf monopoly seems a bit too ominous.
https://watertechbyrie.files.wordpress.com/2014/06/duck.png
Powers the name of the game Chief, lol. Hasn’t the
Climate Con-sen-suss Cru shown us that this is so?
just want to say
Robert Rohde, Zeke Hausfather, Steve Mosher
thanks for this post
read with great interest
trying to remain skeptical of my own skepticism
thanks also to Judith Curry
Then why do we see that WUWT graph of temp adjustments in the GISS data that shows only a warming trend to account for >50% of the current anomaly?
A good check on the accuracy of global warming observation is the bulge in temperature between 1910 and 1940 when the temperature rose by 0.45C and then just as surprisingly fell again. Berkley shows this well. It is explained in my website (underlined above).
Mkkkt: Notse is common In science and every scientist knows how to deal with it. Some are better than others at recognising signals in noise. It depends on experience.
So if I were to use a randomly crap set of algorithms, and attempted to justify them by saying that ON AVERAGE there were just as many spurious ups as downs, and claimed to be doing science, would people fall at my feet and worship my magnificence?
Boston has 62 inches of snow since winter started…?
No… 62 inches in about the last 2 weeks, and counting and 76.5 inches this winter. The only predictions that are coming true are those about global cooling and Old Farmer’s Almanac did that back in 2009 but not by looking at homogenated data.
When dealing with data in such massive amounts random errors will disappear into the statistical wash and are unimportant. The only errors that should matter are the systematic ones. And in my opinion those are best addressed not by tweaking the data – a fundamentally dishonest procedure – but by analysing the data to estimate the size of each systematic effect.
The more data is adjusted the less information it contains. It is neither necessary nor wise to try to detect and correct every breakpoint in the data. That represents a massive overadjustment. The adjustments become so significant that the adjustment procedures themselves risk becoming sources of systematic error. These procedures turn unsystematic random breakpoint errors that should have no effect on the overall temperature record into sources of systematic error.
Furthermore the focus on breakpoints means that real sources of systematic error are ignored. Often it is implicitly asserted that correcting breakpoints will somehow correct these errors too, but that assertion is false. Consider UHI for example – a real and significant sourece of systematic error. Breakpoint adjustment and geographic averaging will not fix a systematic error like UHI. Indeed it is far more likely to amplify it by preserving slow spurious warming trends recorded at instruments in urbanising regions and eliminating the jump corrections caused by station moves to more pristine environments. Geographic averaging will then smear these adjustments to adjacent rural stations further contaminating the data. If your supposed UHI correction procedure leads you to adjust the temperature recorded 50 years ago at a rural station adjacent to where a city will later develop downwards by 2 degrees, then you are quite simply doing it wrong.
Actually Ian, homogenization seems to do a reasonably good job of dealing with UHI, at least in the U.S.
I wrote a paper with the NCDC folks that looked at how well homogenization does at removing UHI in the U.S. record. We looked at various definitions of urbanity based on things like lights visible at night from space, population density, population growth, and impermeable surfaces. We also ran the homogenization algorithm both with all stations and with only rural stations used to detect breakpoints, to avoid any risk of falsely adjusting rural stations upwards. We found that there is a sizable UHI signal in the raw data, and that most of it is effectively removed by homogenization even when only rural stations are used to homogenize, across all the different urbanity definitions examined. ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/hausfather-etal2013.pdf
My fundamental criticism is that things like breakpoint adjustment and geographic averaging, while they are attempts to address real issues, are excessively complicated and therefore likely to cause more harm than good. They destroy information and hide the data in a way that makes it difficult to see what is going on. The fact that you needed to spend such a significant amount of effort to try to understand the effect of these adjustment procedures is illustrative of the problem.
I argued that breakpoint adjustment cannot address UHI and indeed is likely to make it worse. Homogenisation however might be able to address UHI if high quality rural stations are properly identified and pinned. Some kind of geographical adjustment is needed in any case to compare the measurements from differently distributed instruments at different dates.
What we normally see however in most of these records, is a combination of several adjustment procedures – for example breakpoint adjustment and homogenisation and possibly also TOBS. This introduces further complexity. Order of adjustments starts to matter for example. You end up in a place where it is very hard to understand what is going on.
My real beef is with breakpoint adjustment which I believe is excessive and unnecessary. For example many breakpoints are caused by maintaining the instrument or shifting it a short distance to a more desirable location. By eliminating these breakpoints you are effectively undoing these changes which were usually made for good reason. Why do that? Is it even desirable to try to do that?
Bear in mind that the problems that those who maintained or moved the instrument were trying to address often involve gradual deterioration which will not be caught by a breakpoint detection algorithm. While there are countervailing examples of breakpoints caused by sudden changes to the environment – laying concrete and so on – in the absence of any information as to what caused breaks and in particular in the absence of any reason to suspect the the effect of these breaks is to introduce a SYSTEMATIC bias into the temperature record, I would suggest that we would be better off not making any breakpoint adjustments at all.
UHI is not gradual.
1. It is seasonal.
2. it is tied to specfic synoptic conditions.
So if you look at a long term rural verus urban plot, what you’ll see
is certain months or seasons where it’s pronounced and the rest of the year… nothin.
The onset of UHI clearly is gradual as it is determined by the rate of urban growth. Cities don’t string up overnight.
Yes UHI is seasonal. It effects minima more significantly than maxima, we get more of it in winter than in summer, and it is more significant at night than during the day. So …
You have not looked a long term studies of uhi
Steven – I regard your claim of a sudden onset for UHI with considerable skepticism.
For this to be true UHI would need to have a sharp boundary (so that adding a single building would shift the UHI boundary enough to suddenly put an instrument inside it). It would need to have a constant magnitude in all urban areas subject to it no matter how built up so that continued development around an instrument does not increase the size of the effect. Those ideas are at odds with the physics of what causes the effect. I’m just not buying it.
You seem to be claiming to have found this empirically from looking at records? Telling the difference between a step change and a gradual change in a record subject to any kind of noise is very difficult. The noise will make you see steps where there are none. You would need to apply some kind of statistical significance test – eyeballing a graph isn’t good enough to settle this claim. I am extremely skeptical that you have evidence that UHI is a step change (as opposed to a gradual one) capable of passing a statistical test of significance.
Wrong Ian
I am not arguing that the onset is sudden.
Finding gradual change is easy.
See my answer to pielkes challenge on
This thread.
You have not worked with the data.
That shows.
Process data more. Comment less
After reading the post and comments, I am fairly satisfied that the people who are working on BEST data set are sincere and their statements of why they did what they say they did have traceable logic behind those efforts.
The Urban Heat Island effect seems to me to be problematic as with the increase in global population, there has been a population movement from rural to urban that impacts the UHI effect. Adjustment for Mumbai may not be the appropriate adjustment for Rio de Janeiro because all things are not equal.
This leads me to Rob Ellison’s comment:
” Climate models at best define a probability space – there are no unique, deterministic solutions.”
RiHoo8, since you say you’ve read the comments here and are satisified with what BEST has said, may I ask you about this comment I made and the responses I received? Specifically, after reading it, do you have any reason to believe BEST’s estimation of “empirical breakpoints” improve its results?
It is trivially easy to see this step introduces a great deal of spatial smearing. It is also trivially easy to see because of it, BEST has far less spatial resolution than that of other groups. Can you offer any reason that should be desirable?
Whether or not BEST is sincere, and whether or not there is a reason for what BEST does, it seems reasonable to me to question BEST’s “empirical breakpoint” estimations. As far as I know, BEST has never demonstrated these estimations have skill, and they cause BEST’s results to be very dissimilar from previous ones.
And for the record, this isn’t a minor thing. If BEST discarded its “empirical breakpoint” estimations, they would find ~20% less warming and would have far greater spatial resolution.
Steven Mosher, “Why include data you know to be unsuitable.
You’re still wrong. Officially in my ignore bucket.”
It is an reference to compare best and giss to. Best and GISS agree well with that crap in FL but don’t in GA, call it third party noise if you like. btw ghcn/cams on climate explore only goes to 1948 or I would use that.
Thank Ri.
I’m continuing some work on the UHI issue and on improving the fidelity of local predictions. The chief difficulty is getting anyone with funding to believe that is actually an interesting science issue.
If they thought it would help the cause, they would fund it.
Steven Mosher, “I’m continuing some work on the UHI issue and on improving the fidelity of local predictions.”
Then reduce interpolation range with instrumentation density. If you don’t need to interpolate 1000 miles, don’t do it.
Georgia wid LRI
https://lh5.googleusercontent.com/-kkvameZP34k/VNpvnPKYsGI/AAAAAAAAMoA/rd-bXYpIm64/w909-h511-no/Georgia%2BTemperature%2Banomaly.png
Georgia wid out
https://lh5.googleusercontent.com/-puCrObM1GA8/VNqcRQjjjlI/AAAAAAAAMpM/7d6KbJjkAnQ/w910-h502-no/georgia%2Bwith%2Bgiss%2B250km.png
Optimal range interpolation, what a novel concept.
Captain cruTs is unsuitable. Read their docs.
Want to know what Harry read me worked on??
Captain. Interpolation isn’t the issue.
The correlation decides the distance for kriging the weather..
CRU Ts is unsuitable for some things. That chart compares GISS 1200 long range, GISS 250 short range, NCDC and CRU TS. You focus on just one you find “unsuitable”. Typical.
Mosher, “Captain. Interpolation isn’t the issue.
The correlation decides the distance for kriging the weather..”
Interpolation isn’t an issue for “global” it does smear regional. People are picking at you with “regional” issues which aren’t what your product is about. On a “regional” basis, interpolation “adjusts” as much or more than the standard adjustments and in just about the same way, some warm some cool That is unavoidable for a “global” product.
For correlation determining the distance for “weather”. Some areas are naturally out of phase with fairly close neighbors. Paraguay is at 25C which is right on the shifting westerlies line. Admundsen Scott is a corner located in the eye of a nearly perpetual polar cyclone. Even kriging has it limits.
That should be 25 south not C
captdallas2 0.8 +/- 0.2:
I don’t think it’s reasonable to say BEST is only about global temperatures when it goes out of its way to encourage people to look at temperatures on a much finer scale. Regardless, you’re wrong to say interpolation isn’t an issue at the global scale. Interpolation can influence and bias results at any scale depending on the distribution of sampling. An obvious example is how the presence/absence of coastal stations can influence temperature trends.
Of course, that doesn’t mean interpolation introduces problems with BEST’s global results. It just means we can’t automatically rule out the possibility BEST’s results, even on a global level, have problems caused by interpolation.
This isn’t even just about kriging. BEST’s homogenization process compares stations to their 300 nearest neighbors. It will look for such neighbors as far out as 2500km. That means BEST’s “empirical breakpoints” can get estimated using data as far apart as 5000km (2500 km in each direction). I suspect that affects their results more than the interpolation distance for the kriging step.
To be fair though, the amount of weight stations are given in these calculations is dependent upon how far away they are. That means close stations will “matter” more than far ones. The more densely sampled an area (for any given time), the less this issue should matter.
Brandon, Kriging is superior to just long range interpolation, but as stations drop out it ends up being about the same. My biggest issue is where the nearest neighbors are coastal or on the other side of a “weather” boundary. Georgia is a good example because it has both coastal and mountain neighbors so it seems to be adjusted more by interpolation than standard station adjustments.
Georgia btw had a huge land use change with tree farming as did most of the southeast. Coastal and mountain stations would have less of that impact so they seem to negate what should be obvious land use influence.
Since BESTs adjustment method is different and tends to verify the others, I doubt there is much more that can be done there. Anomaly baseline is an issue but mainly for the “warmest” month ever stuff.
Brandon & Mosher,
On the “global” versus “local” subject. You have climate regions. Georgia and the Carolinas have coastal, Piedmont and mountain climate regions so like to like interpolation or kriging would produce a better local climate reconstruction. The plains should be weighted to plains climate etc. That isn’t all that important for “global”. So yes, BEST is actually “globally” oriented and should not be expected to reproduce state climate. That is what state climatologists are for.
captdallas2 0.8 +/- 0.2:
Agreed. I just don’t think that’s as much of an issue right now due to the figures showing the amount of spatial smearing caused by BEST’s homogenization. The difference in spatial resolution between its results with and without their “empirical breakpoint” calculations is enormous even though the same kriging process is used in both cases.
I think BEST’s “empirical breakpoint” calculations have so much influence they drown out most of the concerns about the kriging process. I’d definitely be interested in looking more closely at the kriging process and seeing if it could be improved, but I think any gains we could get in improving it would pale in comparison to those we could get in improving (or even just removing) the “empirical breakpoint” step.
It would be interesting if there were two competing data issues in that area which mostly cancelled out. In such a situation, what would a breakpoint algorithm do?
I think BEST’s adjustment method could be greatly improved, and I think that would greatly improve the BEST results for sub-global scales. On a global scale, I don’t think it’ll make that large of a difference. I still think it matters though.
I don’t think problems with BEST’s work are going to completely overturn anything, but BEST portrays its results as being accurate to less than a tenth of a degree. BEST released a report for the media on 2014 temperatures, paying attention to variance of less than 1%. If variance that small is enough to go to the media, I’d say it is certainly worth examining the adjustments they perform which increase their results by ~20%.
captdallas2 0.8 +/- 0.2:
My problem with this argument is that’s not how BEST portrays their work. They release their results in data files with ~100km x ~100km grid cells. They’ve talked about wanting to get the resolution of those files down to ~25km grid cells. I don’t see how to square that with the idea BEST is globally oriented. If BEST is globally oriented, why is it giving out results for such local scales?
Similarly, BEST’s website has a feature which lets you look up the trend for individual regions, or even individual cities. You can even pull up the results for any part of the planet by clicking on a map. Why would that be true if BEST is globally oriented?
If BEST wants to be globally oriented, I’m okay with that. What I’m not okay with is encouraging people to look at BEST’s results on a fine, local scale then turning around and saying people shouldn’t use BEST’s results on anything other than a global scale.
Though to be honest, I can’t even see why we’d care about BEST if were only supposed to be used at a global scale.
Brandon, it isn’t what BEST wants to be or thinks they are, it is they are limited to what they have.
https://lh4.googleusercontent.com/-nP5pFSG-ynA/VNtJQU1qEyI/AAAAAAAAMpg/sYHnvwCy6ac/w763-h471-no/southeatern%2Bstates.png
That is something close to the “raw” data for the four southeastern Atlantic states. Georgia and SC changed from huge cotton, tobacco and rice states to tree farms, peaches with more limited cotton and tobacco, still a little bit of rice thankfully. Florida rerouted rivers and installed drainage and moved to cattle, cane and potatoes along with planted pines and oranges. The group that should resolve local climate history issues would be the state climatologist and department of agriculture.
btw, you don’t know what hot is until you are chopping cotton or pulling tobacco in the summer.
Brandon, I have another comment in moderation, but here is a quick comparison. Using CRUTS as a “raw” approximation.
SC GA NC FL
trends
CRUTS 0.14 -0.28 0.42 0.68
BEST 0.73 0.65 0.82 0.76
GISS250 0.42 0.06 0.55 0.71
Adjustments from “raw”
BEST 0.59 0.93 0.40 0.08
GISS250 0.28 0.34 0.13 0.03
If GISS250 and BEST station adjustments are about the same its the interpolation causing most of the change in trends. The most “normal” climate for long range interpolation would be coastal which also has greater station density, especially long term. Note how Florida has almost no correction and NC with less coastal exposure is less than SC and GA.
“Globally” it means nothing, just like Paraguay, Alice Springs and New Zealand. Local is a different matter.
Why include data you know to be unsuitable.
You’re still wrong. Officially in my ignore bucket.
Steven Mosher, “Why include data you know to be unsuitable.
You’re still wrong. Officially in my ignore bucket.”
Whatever. CRUTS is looking for things like first frost and extremes which would be important for a local climate record and Best tends to over smooth those kinds of events. While best warms Georgia, it cools the Bahamas and Cuba, while it leaves Florida and New York about the same. that appears to make CRUTS3.22 better in most cases for a “local” climate and BEST better for a “Global” climate.
As far as “global” absolute temperature goes, BEST is cooler than CRUTS3.22 mainly because of the Antarctic which is at an average elevation of 2000+ meters. If you use potential temperature they are about the same.
Robert Rohde, Zeke Hausfather, Steve Mosher – I am pleased you are presenting this in-depth analysis and discussion. However, when will you (and BEST) perform an equivalent analysis of maximum and minimum temperature trends?
Regards Roger Sr.
The data is there. The code is there. and there is a joint paper prolly coming out on a related issue. That’s a Rohde effort with other people.
We have to choose between doing science and de bunking lies.
it’s not always an easy choice.