by Mike Zajko
This post addresses an issue that has been coming up recurrently since the start of this blog. I hope it might be a way to step back and reflect on the nature of science in general, as well as a place where we can think about the methods applied in climate science more specifically. I’ve broken the following down into sections that can be read together or individually. I’m hoping for come good discussion of these and additional approaches to the scientific method.
Summary: The scientific method, if presupposed as a core principle of science, deserves some close examination. Adherence to this method or deviation from it is often seen as a way of demarcating science, or distinguishing good science from bad. However, no agreed-upon formulation of the scientific method exists, and I argue it is more effective to consider science’s methods in terms of Hugh Gauch’s “general principles of scientific methodology”. This approach better reflects the actual practice of science, while staying attuned to the fundamental epistemological questions that scientific methodology should be designed to address. Climate science poses its own set of challenges for the design and evaluation of scientific methodology and it is worthwhile to consider how closely certain dimensions of climate science adhere to the general principles identified by Guach, as well as other formulations. Ultimately, I am interested in the question of what it means to “know” scientifically, and how such knowledge is best obtained.
Introducing Myself
I’m a student of the sociology of science, which means that I am not a practicing scientist (unless you happen to be feeling particularly generous towards the social sciences) but do consider myself relatively informed in several of the fields relevant to the debate here. For the past two years my focus has been on climate science, particularly concerning demarcation practices, or how the borders of climate science are maintained and contested. As such I am not an expert on scientific methodology, and certainly welcome contributions from practicing scientists as well as the more philosophically-inclined. However, I do have a professional interest in definitions of science and their uses in the context of scientific controversy. It is in these circumstances in which the scientific method is typically most forcefully articulated, and while I hope to eventually address other common definitions of science (as provided by Robert Merton, Thomas Kuhn, and the various movements Kuhn inspired) the scientific method seems like the best place to start.
The Myth of a Unified Science
There is a persistent idea that science is defined by its adherence to a certain method or form of reason which provides it with a special means of determining truth concerning the nature of reality. Great scientists such as Albert Einstein are sometimes cited as experts on this method, as masters of science’s winning formula. This myth is useful in a variety of ways, but has its limitations. For one, it presupposes a unity among the sciences which does not exist in practice. Typically, the method presented has a close approximation to that generally used in experimental physics, and it is therefore no surprise that its cited spokespersons are often physicists. This would be fine if all sciences emulated the model of physics or were somehow reducible to it, but despite the efforts of many in the history of the sciences, this has not been achieved. Instead there exist multiple methodologies and strategies for evaluating evidence among the diverse sciences. Many of these share common elements, but often they do not. It is possible to select among them some form of the scientific method, and declare all other methodologies as un-scientific. This has also been done in different ways throughout history to exclude various undesirable practices aspiring to scientific status. But any clear formulation of the scientific method would also necessarily exclude a wide range of practices that have produced a wealth of useful knowledge in their own right, and cannot easily be relegated to the bin of pseudo-science.
The problem lies in our desire to find some essential core in science that defines its nature. Adherence to the scientific method or a certain form of reasoning is not the only such essence which has been proposed, but it is the most commonly cited in public. Philosophers of science have produced countless books in their quest to define or demarcate science apart from other domains, and been unable to reach much agreement on the answer. This should not be confused as supporting an “anything goes” approach to science, where all methods (or their lack) are treated as equally valid, but to argue that any remotely accurate definition of scientific practice must somehow account for science’s disunity. It is surprising how rarely the scientific method in particular has been approached in this way, and I will be repeatedly referring to Hugh G. Gauch Jr.’s Scientific Method in Practice as a useful attempt by a practicing scientist to do just that.
For a lengthier elaboration and justification of the above, as well as a review of various “unifiers of science” I would recommend:
Ian Hacking (1996). The disunities of the sciences. In P. Galiston & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts and power (pp. 37-74). [available on Google books]
The General Principles of Scientific Methodology
The conceptual shift away from a singular scientific method (usually expressed as a series of steps involving theory and hypothesis testing, connected by arrows) and towards some general principles of scientific methodology addresses many of the limitations of the myth of scientific unity given above. It allows us to go beyond asking whether a procedure has been conducted in accordance with the scientific method, and to reflect on how to best apply scientific principles to a given problem. This approach should also force us to confront some core epistemological questions concerning the nature of truth, which do not lend themselves to easy answers. Asking such questions is typically not part of a science education, and many philosophers of science have likewise had little to no scientific training. Hugh Gauch Jr.’s (2003) Scientific Method in Practice is an exception in this regard. Gauch is a common-sense realist who stridently defends the principle of scientific truth against postmodern critiques, but also gives due credit to the various disciplines which address as their object the nature of science – the philosophy, history, and sociology of science. As such he avoids the supposed pitfalls of a relativist view of truth, while presenting an account that largely accommodates the disunity of scientific practice. Although I do not agree with a number of Gauch’s positions, I would certainly recommend his book as a relatively uncontroversial take on a controversial topic, and will attempt to summarize the overall argument below. (All page citations to follow are from Gauch’s Scientific Method in Practice, 2003. Cambridge University Press).
The General Principles of Scientific Methodology
As shown in Fig 1. (Gauch 2003: 2), scientific methodologies for different disciplines are partly similar and partly different.
Science contains both specialized techniques which may be found only within certain scientific disciplines, as well as general methodological principles shared, to some extent, by all. Gauch does not intend to map these methods and classify the sciences, and Fig. 1 should suffice to convey his point. The general principles of scientific methodology include “hypothesis generation and testing, deductive and inductive logic, parsimony, and science’s presuppositions, domain, and limits” (19-20).
See Fig. 1.2 (Gauch 2003: 3). Science’s general principles also come in three kinds, including some unique to science, some that are general principles of rationality, and others (like the principle of non-contradiction) being derived from what Gauch calls the “wellsprings of common sense”. Reason is therefore not the exclusive domain of science, rather, “science is a form of rationality applied to physical objects, and science flourishes best when integrated with additional forms of rationality, including common sense and philosophy” (31). Gauch encourages all scientists to become acquainted with science’s philosophical foundations and critiques, if only so they avoid becoming unreflexive problem-solvers who are unable to address their critics.
The PEL Model of Full Disclosure
“Every conclusion of science, when fully disclosed, involves components of three kinds: presuppositions, evidence, and logic” (xv).
Gauch argues that all scientific arguments or conclusions conform to the PEL model given above, even if some conclusions leave their presuppositions undisclosed. In order to properly assess a scientific conclusion, these three components must indentified and recognized as part of an interrelated whole.
Scientists are often unaware of the presuppositions on which their work is based, and Gauch lists numerous varieties (112-155) including ontological presuppositions concerning the nature of reality, epistemological presuppositions of the reliability of sense data and human language to inform our knowledge of the world, logical presuppositions concerning the coherence of the world, and the applicability of inductive and deductive logic. Many of these can safely be considered as common sense, but they are not provable or testable. Such presuppositions also serve to limit the number of hypotheses under consideration to a finite roster of sensible or testable ones (129), since there always exist wild hypotheses which are effectively ruled out in practice. Therefore, while a set of hypotheses is never truly jointly exhaustive, we can treat it as such out of common sense considerations such as (to use a fairly uncontroversial example) “magic is not a valid explanation”.
Logic combines presuppositions and evidence as a crucial part of scientific argument, and while as far as I know, logic courses are not required for most science degree programs, scientists routinely make use of standard logical axioms and arguments (as well logical fallacies) to make their case. Gauch reviews the fundamentals of several forms of logic, as well as common fallacies. The distinction between inductive and deductive logic is treated as fundamental, with induction acting as reasoning from data to support a model (with varying levels of support), and deduction commonly operating as reasoning from a model or set of premises to what would be expected in terms of observed data (where the truth of the premises guarantees true conclusions). Scientific arguments include both deductive and inductive elements, and “inductive problems often contain deductive subproblems” (191). Both deductive and inductive logic are further broken down into various types, and Gauch spends considerable time on probability theory and the Bayesian and frequentist paradigms in statistics. I think it might be a good idea to devote some time on this blog to inductive and statistical methods specifically, considering their strong role in many climate-related arguments (also including Christensen’s paradigm as championed by Terry Oldberg), but these are topics too dense to begin to address in this post.
Parsimony, Domain and Limits
I’m going to briefly define what Gauch means by the above three terms, since they’re probably the less recognized of scientific methodology’s “general principles”.
Since there are always multiple theories that fit the data equally well, parsimony (sometimes identified as Ockham’s razor) becomes an essential and pervasive principle of scientific methodology, essentially dictating that scientists choose the simplest theory that fits (other criteria to consider include “predictive accuracy, explanatory power, testability, fruitfulness in generating new insights and knowledge, coherence with other scientific and philosophical beliefs, and repeatability of results” [269]). Numerous empirical examples are provided to demonstrate the crucial part parsimony has played in the history of science.
Science’s domain and limits refer to what we can and can’t expect of science. Science cannot explain everything or provide its own ethical requirements, and it cannot prove the presuppositions on which it is based (376).
Falsification
Gauch doesn’t devote much of his book to falsification. However, the topic comes up often enough in the climate debate for me to spend some time on it here.
Falsification tends to be the “one bit of Popper” to which scientists are introduced and end up citing as epistemic justification. Those who cite Karl Popper are probably unaware of the radical implications of his work, his low standing among most philosophers of science (who find his solutions largely unconvincing), or the original philosophical problems that Popper felt compelled to address. The “one bit” which Popper popularized in his search for a means of demarcating science from non-science is the idea of falsification, or that “a proposed hypothesis [or theory] must make testable predictions that render the hypothesis falsifiable… a scientist should give his or her favored hypothesis a trial by fire, deliberately looking for potentially disconfirming instances” (103). Gauch think this is wholesome advice, but no more novel an insight than the modus tollens argument in classical logic. As for the practice of scientists seeking to disconfirm their own theories – he considers it part of honest scientific practice (although I would add that the history and sociology of science demonstrate this practice to be rather uncommon).
A recurrent misunderstanding of Popper is to assume that the outcome of a hypothesis test or an experiment designed to falsify a hypothesis could somehow confirm or support that hypothesis. This is precisely the sort of claim Popper wished to avoid, since he argued that all observations were theory-laden (requiring a theoretical basis with which to ask research questions, collect data, and interpret) and that theories could never be proven or verified. He was also firmly opposed to the suggestion that theories could be judged against one another by the weight of supporting evidence, although one might have a basis for preferring one theory over another based on its specification and the sorts of falsification tests it had passed.
However, Popper’s claim that theories could be falsified (and that through such falsification science could refute false theories) itself ran into serious difficulties and was significantly revised and weakened in Popper’s lifetime. First, all theories are underdetermined by data, that is to say “[f]or any given set of observations, it is always possible to construct infinitely many different and incompatible theories that will fit the data equally well” (83). However, according to similar logic, falsification is never conclusive either, an idea grasped by Imre Lakatos (84). Lakatos noted that many theories (to which we may add anthropogenic global warming [AGW]) do not lend themselves to making falsifiable claims or the sorts of “critical tests” that Popper was fond of using. Harry Collins made a related point in 1970s as a sociologist, arguing that experimental results are never decisive in and of themselves (the argument applying both to replication as well as falsification). A claimed falsification of any theory might actually just be a falsification of one its auxiliary hypotheses, such as those related to the observations being a valid measure of the main theory, or the instruments being calibrated and functioning properly. Lakatos prefers to think of evaluating theories based on comparisons among them of corroboration and effectiveness. As opposed to falsification, Gauch lists a number of lines of evidence including explanatory and predictive power, replication, increasing accuracy, and interlocking evidence as part of an argument that the sciences have an “objective grip on reality” (95-96).
Meanwhile, falsification as part of hypothesis testing remains a powerful principle in scientific practice. Popper’s early “naïve falsificationism” is sometimes substituted by an updated “sophisticated” version, which addresses many of Popper’s critics but does not result in the same sorts of bold conclusions. Even when scientists do not explicitly set out to formally falsify hypotheses, we can often think of their work, in retrospect, as a form of falsification through hypothesis testing. Falsification however, does not operate as decisively as Popper had hoped, and cannot differentiate science from non-science (Popper himself significantly revised the use of falsification as a demarcation criteria for science in his later years). Theories can never be falsified or verified definitively, but they can, ideally, be held accountable to evidence and argument.
Further reading: http://plato.stanford.edu/entries/popper/
http://www.stephenjaygould.org/ctrl/lakatos_prediction.html
The Methods of Climate Science
So what does all of this mean for climate science? What methods and applications of scientific principles are appropriate to answer the big questions of climate change? Unfortunately, climate science suffers from much the same sort of disunity as science in general. While it contains many specialized methods, these are the product of a very diverse set of scientific disciplines that developed in a “pre-AGW” world. I see its unification under a single form of appropriate logic unlikely, because a research question such as, “to what extent are human greenhouse gas emissions contributing to dangerous consequences” immediately opens a whole host of complex sub-problems that would best be resolved using different methods. The use of climate models as evidence also leads to various questions regarding their epistemological standing (although I personally think such models can have their uses, as long as their limitations are recognized).
The IPCC has presented several lines of evidence along with the construction of expert consensus to make its case. Criticisms that the IPCC does not follow the scientific method are valid if this method is defined in its simplified elementary sense, because such a caricature automatically excludes the majority of what is conventionally accepted as scientific practice. However, we can also consider how well the IPCC’s case, as well as alternate conclusions put forward by its critics, conform to some of the principles of scientific methodology given above and elaborated in Gauch’s book. Again, while I do not consider Gauch’s account of scientific methodology as authoritative, it does open up some valuable avenues for discussion – in particular I’m interested in how we might consider various lines of corroborating and interlocking evidence as part of the same argument. Since climate science generally shies of making predictions, the predictive power of AGW theory is hard to evaluate and many of its hypotheses or conclusions are difficult to test, but AGW theory does provide a degree of explanatory power that can be compared to other possible explanations for observed phenomena.
That being said, I do not think such a comparison or evaluation can ever “debunk” AGW or “settle” the science. Even Popper had a rather nuanced take of how scientific agreement results from testing, and science in general proceeds just fine with indefinite conclusions, entertaining multiple hypotheses simultaneously. If there’s anything we can learn from the philosophy of science, it’s that answers to even basic scientific questions do not come easily.