by Judith Curry
Lindzen and Choi have published a new paper entitled “On the observational determination of climate sensitivity and its implications.” This paper is pursuant to a previous paper on the same topic that was discussed by me on a thread at ClimateAudit. The paper is receiving substantial attention in the blogosphere owing to the unusual attention that the paper received by the editors at PNAS.
Lindzen and Choi Part I
Citation: Lindzen, R. S., and Y.-S. Choi (2009), On the determination of climate feedbacks from ERBE data, Geophys. Res. Lett., 36, L16705, doi:10.1029/ 2009GL039628.
http://www.seas.harvard.edu/climate/seminars/pdfs/lindzen.choi.grl.2009.pdf
I ran a thread on this paper over at ClimateAudit on this paper.
The two main findings of LC are that: the sensitivity of the Earth’s climate is much smaller than the conventional estimates (e.g. IPCC); and climate models substantially disagree with observations and produce a sensitivity that is far too high (and hence are producing falsely alarming projections).
I was quite critical of the paper, as were others.
Lindzen and Choi Part II
The new paper (dubbed here “Part II”) addresses the published criticisms.
Citation: On the Observational Determination of Climate Sensitivity and Its Implications. Asian Pacific Journal of the Atmospheric Sciences, in press. [link to complete manuscript]
The manuscript addresses many of the concerns raised by Part I, some of these concerns were addressed more satisfactorily than others. And a whole host of new issues are raised by the paper. The PNAS reviews of the paper can be found here.
PNAS review
The paper was originally submitted to PNAS. The PNAS review history on this paper is of interest, and it is documented ins some detail at masterresource.org. The PNAS has some unusual policies for submitting papers and reviewing them. While PNAS will consider directly submitted papers, nearly all of the successful papers are communicated by a member of the NAS. Each NAS member is allowed to communicate up to 4 papers per year. Members of the NAS have special privileges in this process, whose papers are referred to as contributed. From the PNAS guidelines for submitting papers:
Many of the same principles apply to papers contributed (Track III) for publication. Two or more appropriate experts, free of conflict of interest and not recent collaborators, should comment on both the original and the revised manuscript. All critiques should be returned on the standard PNAS review form.In rare instances, the Board expresses a concern about a Communicated or Contributed paper, most often because the referees are not considered appropriate or the member has not returned a significantly revised manuscript to the referees. If the original referees are not considered appropriate, however, the Board may suggest additional experts. Previously, these experts were anonymous to the communicating/contributing member. In response to concerns that some members have raised, the Board will henceforth suggest potential new referees who will be consulted only with the communicating/contributing member’s approval. In addition, to help the Board decide if a referee has the requisite expertise, PNAS staff will provide a link to that referee’s lab or office website. We ask for your help in providing this information during the submission process. In practice, very few Communicated and Contributed papers are rejected by the Board. Last year approximately 800 Communicated and 800 Contributed papers were submitted, of which only 32 Communicated and 15 Contributed papers were rejected.
Basically, if you are a member of the NAS, you can organize your own peer review by asking two people to review the paper. If the reviewers are deemed unsuitable, the editors will consult with other reviewers only if approved by the contributing author. PNAS deemed Lindzen’s reviewers to be unsuitable, and then recommended 5 potential reviewers which Lindzen found to be unsuitable. The PNAS send the paper out to 4 reviewers, with only two of them deemed to be marginally suitable by Lindzen.
So why the special treatment for Lindzen? In a letter to Lindzen from the editor:
On the other hand, it is precisely the high potential relevance of the issues addressed in the manuscript that requires the most objective and informed peer review conceivable. The article submitted by Lindzen and Choi is a response to strong (published) criticism of a previous paper by the authors. Not being a true specialist in the pertinent field, I cannot provide a solid judgment whether Lindzen and Choi have overcome that criticism. But it is good scientific practice to involve either some of those who have raised the counter-arguments (and may be convinced by an improved analysis) in the review or to solicit at least the assessment of leading experts that have no direct or indirect affiliation with the authors.
Ok, because this paper is skeptical of the consensus, it is deemed to require an unusually rigorous peer review. Hmmm. . .
JC’s comments
The editor is correct in that this is a potentially important paper. With regards to the scientific merit of the paper, the PNAS reviews were actually quite thorough and do not seem unfair to me; they raise many valid concerns. LC chose not to submit a revised version to PNAS and instead went with another journal.
With regards to how the PNAS editors treated Lindzen’s paper, I am of two minds about this.
First, I have been harshly critical of “pal review,” and the PNAS papers contributed by NAS members is the worst form of pal review. Ideally, every paper would be subjected to a rigorous review by 4 people including those who are likely to be critical. It should be incumbent on the editor (often with the advice of associate editors who are more knowledgeable of the subject matter) to sort out any unreasonable criticisms. And by the way, I also think that the reviews and editorial decisions should be made public on the web, such as in numerous online Discussion journals.
Second, PNAS violated its own guidelines in the treatment of the LC paper. Looks like potentially important papers by skeptics get “special treatment”, whereas unimportant and often dubious papers by consensus scientists slide right through. This treatment feeds into the narratives of McKitrick, Spencer, Christy, Douglass and Michaels about unfair treatment of skeptics by the journal editors. The establishment would often respond to such criticisms by saying that these are marginal papers by marginal scientists, and that more reputable and recognized scientists such as Lindzen have no trouble getting their papers published. Well, this PNAS episode certainly refutes that argument.
PNAS needs to decide whether it wants to be a vanity press for members of the NAS, or a rigorous peer reviewed journal. Either- or, with no special treatment for skeptics.
Fair peer review and editorial decisions on a highly controversial and politicized topic is a challenge. I have published one such paper (mixing science and politics) and have another one under review (uncertainty monster). In both cases, upon submission I wrote a letter to the editor explaining the controversy and why I was concerned about a fair review. I provided a list of individuals in one case and a category of individuals in the other case (anyone involved in the AR4 or AR5) that should not be reviewers. The editors cooperated in this. In the case of the mixing politics and science paper, the subject was too volatile even for reviewers judged to be “fair” and the editor had to step in and make the decision in the face of all this.
In the end, it is far more important that controversial papers be published than buried in the publication process. Far better for a flawed paper to be published than for a potentially game changing paper to be buried. LC’s work on this topic needs to be pursued, challenged, and understood.