Climate Etc has discussed the IPCC’s new protocol for alleged errors and an error found by Nic Lewis regarding climate sensitivity. This post discusses another error in IPCC AR4, one that has been around for some time but has only recently been reported to the IPCC.
The IPCC radiative forcing diagram
An important part of the IPCC WG1 report is Figure SPM.2, which represents the radiative forcing due to various human activities. The diagram also appears in FAQ 2.1and as Figure 2.20, and the details of its construction are given in Chapter 2. On the right of the diagram, in red, are the warming effects, such as carbon dioxide, while the cooling effects are on the left in blue.
This post is concerned with the largest of these cooling effects, the “aerosol cloud albedo effect”. Aerosol effects can be split into a “direct” effect, caused by the aerosol particles reflecting sunlight, and “indirect effects”, relating to the effect of aerosols on clouds, leading to a larger number of smaller water droplets. There are two indirect effects: first the “albedo” or “Twomey” effect, which is that more smaller particles are more reflective, and second the “lifetime” or “Albrecht” effect, that smaller particles take more time to accumulate and form rain, so clouds last longer. The physics of this is complicated, and this is an area where the level of understanding is categorised as “low”. For the purposes of this post, a detailed understanding of these processes is not required.
The first curious point about the IPCC’s assessment is that the second indirect effect is “not considered as RFs” (radiative forcing). This claim is contradicted by many of the published papers which clearly regard the second indirect effect as a radiative forcing. The combined indirect effects are listed in Table 2.7, but the high-impact figure in the SPM and elsewhere only shows the albedo effect.
Note that the RF value given by the IPCC for the cloud albedo effect in Figure SPM.2 is -0.7 W/m^2.
The IPCC error, noticed in 2008
The IPCC bases its value of -0.7 on the literature summarised in Table 2.7 and Fig 2.14. [For the purposes of this post, let’s assume that the IPCC has included all relevant papers and reported their results correctly, ignoring for the time being the fact that the IPCC refused to cite a relevant paper by Morgan et al 2006, as recently discussedhere.] After some discussion of these papers, the report says on page 180,
“Based on the results from all the modelling studies shown in Figure 2.14, compared to the TAR it is now possible to present a best estimate for the cloud albedo RF of -0.7 W/m^2 as the median, with a 5 to 95% range of -0.3 to -1.8 W/m^2.”
The IPCC error was first noted in a contribution to Climate Audit by Michael Smith, back in January 2008. He pointed out the median of the data shown in figure 2.14 is not -0.7, it is actually -0.985. Note that one of the numbers from the Williams et al study listed in Table 2.7 is not included in Fig 2.14. It is not explained why this number was omitted; perhaps it was thought not to be an independent estimate. If this number is included, the median is -1.07. On the CA unthreaded post there is some further discussion involving UC and RomanM (two smart people with a track record of reverse engineering climate science claims), who are unable to understand how the figure of -0.7 was obtained. Michael posted the question at RealClimate, where their Martin Vermeer was also unable to explain it (see comments 123 and 127 on this thread.)
Reporting the error to the IPCC
I reported this error, along with two minor typos, to the IPCC secretariat, as follows:
I write to report an error in the IPCC AR4 WG1 Report, Chapter 2, page 180. It is stated that the median cloud albedo RF from the results in Figure 2.14 is -0.7 W/m2. In fact the median is -0.985.There are three further very minor errors in this section:
1. Table 2.7, Chen study, 0.54 should be -0.54.
2. Table 2.7 note d, reference to Fig 2.16 should be 2.14.
3. Fig 2.14 omits one of the bold values from Table 2.7 (Williams et al study). If this is included the median is -1.07.
I was interested to see whether the IPCC would correct the final value from -0.7 to -0.985, or would try to maintain the final value of -0.7 and change the text to explain where the number came from. I expected the latter.
The official response came a month later. After some introductory material, it stated:
The Authors have confirmed that the cloud albedo forcing estimate is based on the subset of model studies in Figure 2.14 with more complete aerosol species representation. This estimate does not double-count similar forcing calculations within the same study. These two points may help explain the differences between your own calculations and those of the Chapter. It has further been determined that the section text does explain the reasoning behind the forcing estimate adequately. While the text could have been even more explicit on this, the present text is not in error, and therefore does not require changing.However, it has been concluded that the Radiative Forcing value listed for Suzuki et al. (2004) in Table 2.7 on page 174 was incorrect. The value should be -0.54 and not 0.54 as listed.
In addition, footnote d of Table 2.7 on page 176 should reference Figure 2.14 and not Figure 2.16 as given.
These two errors have been listed in an updated WGI AR4 Errata which is available from the WGI website (https://www.ipcc-wg1.unibe.ch/publications/wg1-ar4/wg1-ar4.html.) and attached for your convenience. We have asked the IPCC Secretariat to update the errata on their web site as well and this typically takes a few days.
Thank you again for bringing these errors to our attention and for your interest in the work of the IPCC.
Best regards,
IPCC WGI TSU
So apart from acknowledging the two typos, the IPCC denies that there is any error at all. The claim that “It has further been determined that the section text does explain the reasoning behind the forcing estimate adequately” must rank as one of the most absurd claims of the IPCC.
How the value of -0.7 was achieved
The IPCC response still does not fully explain how the value of -0.7 was arrived at, but here is my attempt to derive it. Starting from Figure 2.14, ignore the upper half of the diagram (not explained in the text). Then combine results from different computations within the same study, despite the fact that the caption to the Figure says explicitly that they can be regarded as independent. Let’s suppose that we take the median within each study. This leaves the following numbers:
[-0.45 -0.85 -1.85 -1.35 -0.54 -0.52 -0.77 -0.5 -1.1 -0.68 ]
The median of these numbers is -0.725, which with rounding could be interpreted as -0.7.
Looking back at the second order draft
It is often useful to look at the Second Order Draft (SOD) of the IPCC report. Recall that the SOD is the last version to be inspected and commented on by expert reviewers before the final report is produced.
Looking at pages 41-42 it is interesting to see that the results are summarised correctly: “the average RF from all models in Table 2.7 is -1.09 W/m^2”. Later, on page 45, this number is revised downwards:
“It is possible now to present a best estimate for the RF of (-0.9 +- 0.43) W/m^2 as mean and standard deviation, based on the results from the 8 modelling studies shown inFigure 2.16: Bottom. These models represent the complexity of the aerosol-cloud interactions, in terms of the species included, the state of the mixtures, etc., to the best current knowledge in forward calculations.”
The differences between the SOD and the final version raise some awkward questions for the IPCC.
- Why was the statement that the average cloud albedo RF was -1.09 in the SOD deleted for the final report?
- Why was the clear explanation that only data from the lower half of the figure was used dropped in the final version?
- Why was the average changed from a mean (in the SOD) to a median (in the final report)? (The answer is presumably that it gives a smaller number, but it can be checked that no expert reviewer asked for such a change.)
Note that two more studies were included in the final report, Chen and Penner 2005, median value -1.1, and Penner et al 2006, median value -0.68, so the introduction of these cannot be responsible for changing the overall number from -0.9 to -0.7.
A flaw in the IPCC process
This incident illustrates a serious flaw in the IPCC process. After the second round of reviewer comments on the SOD, IPCC authors are free to insert whatever they like into the final version of the report (in this case, further tricks to make the number smaller, and an arithmetic error). Ross McKitrick noted this flaw in his recent report on the IPCC, and gave two examples. Another example is the insertion into the final report of misleading short and long trend lines on the figure in FAQ 3.1.
Summary
The IPCC chose not to regard the second indirect effect as a forcing, and carried out three distinct fiddles in order to make the headline figure for the cloud albedo effect as small as possible.
1. Use only data from the lower half of Fig 2.14 (explained in SOD, not explained in final report).
2. Combine different values obtained within the study into one (contradicted by caption to Fig 2.14 which states they are independent).
3. Switch from mean values (in SOD) to medians (in final report).
Each of these fiddles reduces the number by about 0.1.
The attempts by the IPCC to disguise this sequence of fiddles led to an erroneous statement in the final report.
When the error was reported, it was denied by the IPCC, with the false claim that the text explained the reasoning behind the estimate and that there was no error.
JC note: Paul sent this post to me today via email, unsolicited. This is a guest post, and the views expressed are those of Paul Matthews. This post does not imply any endorsement by me of the content (but I sure as heck want to hear what the IPCC has to say about this).
Moderation note: This is a technical thread that will be strictly moderated for relevance and civility.