by Judith Curry
The Berkeley Earth Surface Temperature (BEST) project is giving us much to discuss. A number people have criticized the BEST PR in terms of posting the papers before the peer review process has completed, issuing a press release, etc.
Last February, the BEST project was announced publicly, and I did a blog post on the background and rationale for the study. At the time, this didn’t generate too much interest or controversy; people on both sides of the debate seemed to think this was a good idea.
The BEST group engaged extensively with the other groups producing surface temperature data sets and also with interested skeptics and auditors (e.g. Watts, Mosher, Zeke).
In March, Richard Muller testified before the U.S. Congress regarding the BEST project, I posted on this [here]. Muller took heat from both sides.
Since summer, people have been asking on this blog: “What’s going on with BEST?” ”Why aren’t we hearing anything about BEST?”
BEST’s recent PR
A few weeks ago, I became aware that the BEST data set was nearly ready for public release and that a press release was being prepared. I was not involved in any of the deliberations surround this; this was merely a “heads up” saying that I may be contacted by journos.
On the two recent BEST threads, and elsewhere across the blogosphere and media, many are criticizing the BEST project particularly with regards to their PR: issuing a press release before acceptance of the papers for publication, and the public statements made by Muller. Of the discussions of these issues on the Climate Etc. threads, I have found comments by MattStat to be insightful:
The BEST team has drawn some heat for putting their drafts on line prior to peer review and publication, but I think that your selection of critiques displays the wisdom of their (and, by extension, your) approach. Whatever the results of peer-review might have been, this has been in fact a very severe review by talented experts who were not likely to have been selected as reviewers. You’ll recall that the paper in Annals of Applied Statistics by McShane and Wyner had critiques by 9 teams of colleagues as well as a rejoinder, that paper was put on line well in advance of publication, and the authors referred to the online critiques as well as published critiques in their printed rejoinder and supporting online material (the published version alone amounted to 110 pp.)
I think that Dr. Muller had it right: this is a modern-day version of circulating a preprint while the paper is in review. The published version is likely to be much improved, even if not every critique can be resolved to the satisfaction of all involved.
From another MattStat comment:
As to “media blitz”, I do not think it credible that they might have put up their 4 papers and their data and code and then said (as I would have recommended them to say) “No comment” to all media inquiries. Anything they might have said, and all of their silences, would have been attacked from left and right, from believers, skeptics and deniers. Of the zillions of imperfect ways to handle the media for this politicized topic, they chose one of the imperfect ways.
JC’s Monday morning quarterbacking
The reason I think a press release was appropriate for the release of the BEST data is aptly stated by Steve McIntyre:
The entry of the BEST team into this milieu is therefore welcome on any number of counts. An independent re-examination of the temperature record is welcome and long overdue, particularly when they have ensured that their team included not only qualified statistical competence, but eminent (Brillinger).
Whatever the outcome of the BEST analysis, they have brought welcome and fresh ideas to a topic which, despite its importance, has had virtually no intellectual investment in the past 25 years.
The data set itself is of importance, the notion of public availability and complete transparency is important (particularly in the wake of Climategate and the issues with the CRU data set), and the idea of a fresh look by experts outside of the climate field is important. If I had been in charge of the PR for BEST, this is what I would have emphasized.
Muller chose to emphasize the actual scientific results as described in 4 manuscripts that are in the early stage of the peer review process. I am totally supportive of making submitted papers public available for public scrutiny. However, I am not so supportive of overselling the results of the papers. As I stated in my original post on this, I felt that two of the papers (AMO and surface station quality) were not yet ready for prime time.
I was contacted by a few journos last week, I made my points, but they were interested in the implications for trend analysis, UHI effect, station quality. I made the point that these were complicated issues, and that I regarded the BEST papers (which were as yet unpublished) to be the first of many analyses on these topics using the new data set. This wasn’t what the journos found interesting, and I don’t think any of my quotes on this made it into print.
The issue for which Muller has taken the most heat is his public statements regarding his analysis as being the end to climate change skepticism. To Muller’s credit, he has taken the initiative to engage with and listen to skeptics, albeit on the narrow topic of the surface land temperature data record. Relative to the broader issue of attribution, which are at the heart of skeptical concern, details of the surface temperature record don’t play a terribly large role in most people’s skepticism about climate change.
My bottom line assessment is:
- a press release on this was warranted
- I applaud making the submitted papers publicly accessible at this time
- the spin on the press release and Muller’s subsequent statements have introduced unnecessary controversy into the BEST data and papers