by Judith Curry
I stumbled across this essay by Michael Nielsen entitled “Science Beyond Individual Understanding,” which I think is very relevant to the climate problem.
Hidden knowledge is a concept derived from a paper by economist Frederick entitled “Use of Knowledge in Society.”
The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form, but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. Or to put it briefly, it is a problem of the utilization of knowledg not given to anyone in its totality.
Nielsen’s essay adapts the concept of hidden knowledge to science:
In economics, many basic facts, such as prices, have an origin which isn’t completely understood by any single person, no matter how bright or well informed, because none of those people have access to all the hidden knowledge that determines those prices.
By contrast, until quite recently the complete justification for even the most complex scientific facts could be understood by a single person.
Science is no longer so simple; many important scientific facts now have justifications that are beyond the comprehension of a single person.
JC: This is certainly true of the climate problem in all its complexity.
No single person understands all of this, except in broad detail.
Instead, there will be a large group of people who collectively claim to understand all the separate pieces that go into the discovery, and how those pieces fit together.
JC: This would explain the IPCC “consensus” and the large number of people that support it.
Two clarifications are in order. First, when I say that these are examples of scientific facts beyond individual understanding, I’m not saying a single person can’t understand the meaning of the facts. I’m talking about a deeper type of understanding, the understanding that comes from understanding thejustification of the facts.
JC: Yes, this aptly describes the missing arguments and justifications from the IPCC report, which instead substitutes citations of papers and lists of evidence, summarized by a confidence level derived from expert judgment.
Second, I don’t mean that to understand something you need to have mastered all the rote details.
JC: The emphasis of a certain element of the climate blogosphere on factoids and information and the dismissal of anyone who makes a misstatement is not useful in the context of understanding.
How do we know whether [scientific discoveries are] right or wrong? The traditional process of peer review and the criterion of reproducibility work well when experiments are cheap, and one scientist can explain to another what was done. But they don’t work so well as experiments get more expensive, when no one person fully understands how an experiment was done, and when experiments and their analyses involve reams of data or ideas.
JC: Because of the complexity of many climate studies, peer review is typically cursory in its ability to uncover methodological or logical problems.
Might we one day find ourselves in a situation like in a free market where systematic misunderstandings can infect our collective conclusions? How can we be sure the results of large-scale collaborations or computing projects are reliable? Are there results from this kind of science that are already widely believed, maybe even influencing public policy, but are, in fact, wrong?
JC: need I actually say it?
But such collaborations will be no good if we can’t assess the reliability of the results. And it would disastrous if erroneous results were to have a major impact on public policy. We’re in for a turbulent and interesting period as scientists think through what’s needed to arrive at reliable scientific conclusions in the age of big collaborations.
JC: this seems to be what Steve McIntyre is going after, in terms of the auditing concept and engineering-style explanation, and making explicit the “hidden knowledge”.
p.s. more from Michael Nielsen this weekend.