by Judith Curry
This week I am attending a meeting of the Earth Science Subcommittee of the NASA Advisory Council. As described in the Public Notice for the meeting, the topic of this meeting is evaluation of NASA’s Earth Science Modeling and Activities.
The ppt presentations from the meeting are stored temporarily in a drop box, they will remain at this site for 14 days, so download anything you would like a permanent record of.
Overview from Mike Freilich
Michael Freilich is Director of NASA’s Earth Science Division. A few points of interest from his overview presentation (ESDUpdateFreilichESSMay11):
As per verbal statement, NASA is the source of 48% of the funds for the USGCRP (U.S.Gobal Change Research Program).
The launch of the Glory satellite failed for the same reason as the OCO (carbon dioxide) satellite. The issue is Taurus-XL launch vehicle, which has crashed 3 out of its last four launches. The problem is that there aren’t any other certified U.S. launch vehicle for this range of payload weights.
The Glory satellite was to carry a TIM (Total Irradiance Monitor) and APS (Aerosol Polarimetry Sensor). Unlike the OCO mission that failed, a copycat version of Glory will not be developed. Re TIM, the SORCE and ACRIMSAT missions will continue to at least 2016, so solar radiation measurements are covered. Re APS, this sensor is being reassessed by two separate review panels.
An interesting tidbit: Glory was classed as a “C” priority mission (the lowest priority that actually flies) with a class 2 launch priority (3 is top priority with the best launch vehicles). If all satellites were class 3 priority A, the program would be unaffordable. Whether the ASP or some version of the polarimeter survives, remains to be seen.
A “launch vehicle crisis” was discussed: “ESD/SMD/NASA is losing reliable, predictable, access to space via affordable, proven launch vehicles. After 2 consecutive failures of the Taurus-XL LV, there is no certified U.S. LV with capacity between the Pegasus (440kg to LEO) and the Atlas-V (9750-29,240 kg to LEO). LV availability and reliability problems are causing launch delays and cost increases now, and will have greater impacts on the Earth and Space science programs in the coming decade.” The issue is the high cost of private sector launch vehicles, the requirement to use U.S. launch vehicles (to support the development of a viable private sector launch vehicle capability in the U.S.), and a mismatch of private sector practice (continued improvement of vehicles) vs NASA’s certification requirements.
The GISS climate model was presented by R. Miller, pdf link here. A few comments:
The GISS model is gradually starting to more closely resemble the other U.S. climate models. For CMIP5 (AR5), they have increased horizontal resolution to about 2 degrees. They are using the NCAR/DOE sea ice model. Their aerosol-cloud interaction (aerosol indirect effect) is far more advanced than the (non existent) aerosol indirect effect in the NCAR climate model. The GISS model is conducting the complete suite of CMIP5 simulations.
One of the committee members asked what kind of outside advisory group provided advice and evaluation of the GISS model. The response was that there is a very large number of people on the model development team, and that they write a new proposal every 5 years for renewed funding, with peer and programmatic review. Note the NCAR climate model has an extensive advisory committee structure. The NOAA GFDL model received a comprehensive review a few years ago; I can’t find the details of this, but when I was a member of the NOAA Climate Working Group, there was a ~2 day on site review that the CWG participated in, as well as a designated review panel that wrote an extensive report.
Michelle Reinecker’s presentation is ESS May2011 Reinecker.pdf in the dropbox. NASA’s other global model is GEOS5, which is a completely separate development from GISS. It focuses on higher resolution shorter time scale simulations, with a particular focus on the assimilation of satellite data. Their new atmospheric reanalysis MERRA is advertised as being comparable to the ECMWF Interim Reanalysis, with a particular focus on the atmospheric hydrological cycle. I use the ECMWF product, but this definitely motivated me to take a look at MERRA.
Discussion with Edward Weiler
Edward Weiler is Associate Administrator for the Science Mission Directorates. We had an hour of discussion. He provided his perspective on what is going on on “the hill.” The importance of international coordination of space missions was emphasized, with shrinking budgets not only in the U.S. but also Japan. The critical issue of the polar orbiters was discussed.
Several committee members (myself included) mentioned the importance of increased emphasis on the applied science program to demonstrate the societal value of the satellite data. Quick availability of the data, realtime data handling is critical for the satellite data to be used for applications.
Evaluating CMIP5 Simulations
Duane Waliser gave an excellent presentation, see Waliser ESS/NAC PPT
in the drop box. The presentation is about making better use of satellite data for climate model evaluation. In particular, a new effort is underway to bridge the gap between the satellite data sets and people evaluating climate models, the bridge consisting of identifying, formatting, arciving, and delivering observations in a form useful for model analysis (which requires model, observation, and IT expertise). Basic tenets of the activity are:
• To be carried out in close coordination with the corresponding CMIP5 modeling entities and activities – in this case PCMDI and WGCM.
• To directly engage the observational (e.g. mission and instrument) science teams to facilitate production of the corresponding data sets and documentation.
The current problem is that there are numerous instrument specific data sets for each measured variable, in inconvenient formats. This project should be a huge boon for climate model validation.
Additional presentations can be downloaded from the drop box:
- Carbon Modeling: C. Potter ESSPotterMay11
- Atmospheric Chemistry Modeling: A. Douglass ESS May 11 Douglass presentation
- Regional Assessment Modeling: C. Rosenzweig ESS C Rosenzweig May 12 11
- Land Information System: C. Peters-Lidard ESS_Peters-Lidard_12May.pdf
A subcommittee member Rober Schutz asked about model verification and validation (V&V), which was the topic of a previous Climate Etc. post). The response was that there is extensive model verification, but the climate modelers seemed unfamiliar with verification in the context of V&V.
NASA is no stranger to V&V; in fact they have an IV&V Facility, with the following mission:
Welcome to the NASA IV&V Facility, home of the NASA IV&V Program. The NASA IV&V Program has embarked on a process to establish an increased value-added/needed presence within the NASA community. The process centers around the NASA IV&V Program’s main purpose of offering needed software services, including IV&V of critical software under development, systems engineering support, and software assurance research.
The NASA IV&V Program provides confidence and integrity in software that cannot be found elsewhere.
The NASA IV&V Program will reduce the inherent risk in the Agency’s ability to procure, develop, deploy and operate software within desired cost, schedule and performance goals by
- Performing IV&V on safety and mission critical software
- Providing software expertise to the Agency’s SMA activities
- Conducting research that improves IV&V and software assurance methods, practices and tools
- Performing Science, Technology, Engineering, and Mathematics (STEM) outreach
- Performing management and institutional services with excellence
Well, needless to say, NASA’s IV&V does not occur in the Science Directorate at NASA, but rather in the context of mission development. But as a result of their IV&V heritage, NASA may be more open than other modeling centers to considering some sort of formal V&V for its climate models.
As per Jack Kaye, NASA’s lead person on the USGCRP, further motivation for this is coming from the USGCRP’s need to consider the Information Quality Act in its forthcoming assessment reports. As per the discussion, USGCRP was planning on relying heavily on peer review in this regard, but I don’t think that is going to satisfy the lawyers and OMB. The Information Quality Act, in combination with the EPA endangerment finding and the USGCRP’s assessment process, may provide the impetus for a more formal and thorough climate model V&V.
This topic was discussed for almost an hour, not sure if/how NASA will respond, but the issue on their table.