My Take on the Scientific Method (Basic Concepts)

Mar 08 2007 Published by under General Science

This is my meager contribution to the "Basic Concepts" series that is going on around here. (I hope to do more later, but for right now I want to start with this one.) I've written about this before at my blog's old location, but I'm writing this now without looking back at that; we can compare later to see how consistent I am.

I also expect other scientists to have a slightly different take on this.

In three words, my view is that the scientific method is nothing more than Applied Common Sense. Now, "Common Sense" has at least three meanings. The first meaning is "the title of a tract written by Thomas Paine," and is not really relevant here. The second meaning is "what seems obvious to people in everyday life," and is very much not the scientific method, and indeed is often at odds with the scientific method. What I'm talking about here is just common sense in the sense of "apply logic, be careful, ask hard questions when something sounds odd." The most important point of this is that the scientific method is not some holy rite that is written, learned, and followed ritualistically by scientists. Indeed, it is something anybody can do in almost any situation.

The scientific method also isn't the clearly delineated set of steps you learned about in junior high school and high school science classes. Those were the steps that start, 1: formulate hypothesis. 2: design experiment. 3: take data. 4: compare data to predictions of hypothesis. Yes, in fact, we are always doing all of these steps, but it is very, very rarely that we do them in the clean, step-by-step method that you learn about in school. Often, we're stumbling about in the dark. We start looking at or exploring something to test one thing, but see odd behavior; we modify our hypotheses or form new ones, and slightly modify our experimental procedure. There's a constant feedback going on. We're playing, but we're doing it carefully, and we're doing it systematically.

The image below sums up my view of the scientific method in its full glory:

scimeth-2.png

One thing to notice in here: the word "fact" is a small little brick down in the corner. When people say "Evolution is a theory, not a fact," they are trying to undermine evolution, but in fact the very statement makes no sense. Strictly speaking, yes, evolution is a theory... but in science, a fact is a small, low-level thing that doesn't really convey much deep understanding. A fact is a datum. "At 3:30PM with my experiment set up such-and-such, my voltmeter had this reading." That's a fact. Or, "At such-and-such a time on such-and-such a day, this instrument measured 25,000 photoelectrons when pointed at such-and-such a galaxy." Another fact. It doesn't tell you a lot, and requires thought to do anything with. But it's the data, the basic simple truths of science.

How do we turn that truth into understanding? First, there is the basic interpretation of the data. Do you understand how your instruments work? Are there things that could be giving you what looks like one result, when in fact something else is going on? How precise, how well understood, is the measurement we've made? Once we've done all of that carefully, we have our experimental result. Experiments can, in fact, be wrong! The facts aren't wrong, but it is possible to make mistakes in our interpretation of the fact. Sometimes, when we state the facts, we'll state them wrong; "this resistor had this voltage across it" may be what we say, but what we really mean is "this voltmeter gave this reading." Perhaps the voltmeter wasn't hooked up as we believed it was. Good experimentalists usually do things right, but we all make mistakes. Sometimes, even when we do everything right, there's something going on that we weren't or even couldn't be aware of. William Herschel wasn't aware of dust in our Galaxy, and so his star gauging placed us closer to the center of the Galaxy in his model than we really are. He didn't do anything wrong, he just didn't know everything that was out there. His star counts were right, but his final conclusions (i.e. his drawing of the Galaxy) were not.

Once we have our experimental results, now we set to the task of trying to digest them, trying to understand them, trying to put the min context and turn them into something that we might call an understanding of some part of nature. The ultimate goal is the big thing at the top: a good, working theory. A theory is just what it says there: a framework for understanding how nature works, and a means of predicting the results of experiments and observations. A good theory works and is supported by the data, but also gives us some sort of overview or picture of what's really going on.

There are wrong or discarded theories. However, in science, a theory is not simply speculation. In common parlance, it is, but that's not what it is here. Facts are the low-level, small things; theories are the big things, the goals.

When do we consider a theory to be "right"? If we are to be completely pedantic, never, really. However, when we have a theory that has explained a huge number of observations and has withstood myriad tests, it becomes conventional to view that theory as right. Certainly, applied common sense would suggest that any new hypothesis that contradicts the theory, or even an experimental result that seems to contradict the theory, should be approached with great caution. Biological evolution is one theory that scientists accept as "right," because it explains so much, and because it has stood up through an explosive increase in biological knowledge during the last century or so. Newton's theory of gravity is right where it works, but we recognize that it's not completely right, that it's only a special case of our deeper theory of gravity (General Relativity) that applies under certain circumstances.

Are we ever getting to fundamental "Truth"? That's perhaps a philosophical or metaphysical question rather than a scientific question. Science is more practical. When a theory works, and keeps working, we consider it right; for all practical purposes, it is. Truth is that which science hopes to asymptotically approach.

Somewhere below theory is the "model." A model can mean a few different things. A purely empirical model doesn't attempt to explain or provide deep understanding, it simply describes what is. Kepler's Laws are three laws that describe how planet orbits in the Solar System work. They are purely empirical laws; hey guys, Kepler says, planets are orbiting in ellipses! Check it out! It doesn't explain why, it merely describes what is. It's useful, because it can predict things. Newton's gravity comes along and says, well, if there is a universal force between every pair of masses that drops with the square of the distance, you get elliptical orbits. Newton's theory of gravity is deeper.

Sometimes a model is something born out of a theory. We have the theory of stellar evolution, but then individual theorists build their models for stellar spectra out of that theory; it is those models that get compared to data. Different theorists have somewhat different models. Partially, this is because the theory is not perfectly understood. Partially, this is because the calculations are monstrously difficult, and while the basic theory is rock-solid, it's tough to figure out what approximations are safe to make in order to render calculations tractable.

Sometimes a model is deeper than just a functional form that some data follows. Sometimes, a model is like a mini-theory. It may be because it's not what we think is really going on, but it's very useful. Or, it may be deliberate fiction that still has useful calculational power (e.g. the "celestial sphere" model used to describe motions in the sky). Sometimes, we call it a model because either it's not broad enough to get the full name theory, or because we're not sure enough of it yet to really want to call it a theory. There is this thing in astronomy called the Unified Model of Active Galactic Nuclei, that indicates that a wide variety of nuclear activity we see in galaxies are all really the same thing, just viewed from different angles. That's more of a model than a theory, because it's not quite broad enough, not quite fundamental enough to really be a theory. But it's more than a simple functional form to describe data, because it does describe to a greater or lesser degree what we really think is going on.

All of this verbiage should convince you that there is not a very clean, delineated line between what one calls a model and what one calls a theory. Models that grow up and start to sound like truth to those not concerned with fine philosophical points start to get called theories. Some people still call the Big Bang the "Big Bang Model," but I call it a theory nowadays.

Of course, the very term theory means two or three different related things, but that's a topic for another post.

What is the scientific method? In a nutshell, the scientific method is (a) carefully analyzing the facts to construct a coherent experimental result, trying to take into account anything that could make the facts appear different from what they are; (b) building models either ab initio to describe data, or from a theory; (c) testing those models against the data, and modifying them if necessary; (d) developing theories that either predict or naturally give rise to working models, or that produce new models to use in step c; (e) forming hypotheses to test in step a either by testing something that a theory or a model says should be there, or by testing something that a theory or a model says should not be there.

Importantly, though, it's not the clean set of steps you learned in high school. All of these steps are feeding back to each other and are going on in parallel. The process can start pretty much anywhere. Sometimes, we just take new data because we can. Hey! A new detector technology! Let's slap it on a telescope and see what we see! That kind of thing has led to all sorts of discoveries. Experiments don't always start with theoretically motivated hypotheses.

None of this is mystical, none of this is even terribly deep. It's not a complicated process that takes years of schooling and a PhD to learn. Anybody can can use and apply the scientific method, and anybody can do it well.

18 responses so far

  • Ambitwistor says:

    I like to refer people to "The Myth of the Magical Scientific Method" by Terry Halwes whenever I run into someone who holds too dearly to their 7th-grade definitions of science, and insists that if you don't start out with a hypothesis then it's not science, or some such.
    (My other pet peeve is people who claim that science which is based on observing natural phenomena isn't "real science", because you can't repeat it in a lab, and they were taught in 7th grade that science has to be "reproducible". Creationists are the most notorious, but far from only, example of this.)
    An accompanying essay, "No Special Method is Required", gets to the heart of your concluding paragraph.

  • CCP says:

    "The only requirements of scientific method are honest observation and accurate logic."
    -R.M. MacArthur, Geographical Ecology

  • Rob Knop says:

    "The only requirements of scientific method are honest observation and accurate logic."
    -R.M. MacArthur, Geographical Ecology

    And funding. Lots, and lots, and lots, of funding.

  • Matt says:

    Well said indeed, I'm going to make sure everyone at work sees this.

  • ssmidt says:

    In three words, my view is that the scientific method is nothing more than Applied Common Sense.
    Somewhere in his book "This is Biology," Ernst Mayr quotes TH Huxley as saying science is "nothing but trained and organized common sense." Mayr goes on to disagree, "Alas, this is not true. Common sense is frequently corrected by science."
    I use these quotes to start discussions on the nature of science in introductory science classes. I think Mayr misses the point of trained and organized (or applied) common sense. Scientists test what common sense tells them, I think that is the important difference.

  • Rob Knop says:

    Ssmidt-- I agree. Indeed, the "trained and organized" is very important. All of that stuff is why I tried to distinguish between different sorts of common sense above. Some sorts of "common sense" might better be called "folk wisdom."
    -Rob

  • incze says:

    I disagree. Science (at least in my opinion, and I'm not a scientist) banishes emotions and sustaines scepticism. That's not common sense. This method isn't appropriate to conduct everyday life, but can be good to investigate systematically well distilled and delineated problems, when decisionmaking is not too urgent.

  • CaptainBooshi says:

    It really doesn't help that we scientists never use the terminology correctly either. We also use the word theory when talking about speculation, which is correct, but confusing. We also use the theory to describe things that aren't theories yet, but just models, like string theory. It's interesting, it could be true, but it hasn't really earned the title of theory yet.

  • The diagram is good, so far as itr goes, but somewhat solipsistic. Where is:
    "write paper",
    "submit paper to journal or conference",
    "referees review",
    "editors make decision based on referees",
    "feedback to author(s)",
    "rewrite/correct paper",
    "paper published/presented at conference",
    "other people read paper/hear presentiaion",
    "other people provide feedback to author(s),
    "attempts to replicate results in other laboratory",
    and the like? You know, the communicative and social side of science?

  • Patness says:

    My personal favorite on that was very recent, and posted on another thread. It went something like:
    "The scientific method is nothing more than a system of rules to keep us from lying to each other." Now, of course, that's not entirely true since even if we didn't lie to each other, there would still be the possibility to err. To expand it, then, the scientific method is Judge Dredd in bloodlust; quickly dispatching those things that are apparently false, and thoroughly scouring all other things, pending their swift execution the moment their falsehood becomes apparent.

  • Patness says:

    What the hell, double-dip. I could rephrase that as "The scientific method is Judge Dredd in bloodlust; everyone is guilty until shown innocent. Execution is stayed only by the presence of evidence in favor of the innocence of a defendant, and such evidence is subject to criticism until the defendant is found guilty."
    - because the former description allows that we make stay for those claims for which there are no evidence or can be no evidence. That's not so; we disregard untestables out-of-hand, and those things that are testable that have no evidence in their favor are not accepted until said evidence is found.

  • Rob Knop says:

    Jonathan -- all of that is in the green arrows. All of those things you talk about are really details of the process.
    Nothing in what I have indicates whether it's an individual or a group or even a set of competing groups doing all of it. It's just the steps that are taken.
    -Rob

  • Rob: I agree. I thank you again for an even more elegant compression than I first realized.
    [insert joke about Comic Book characters Green Lantern and Green Arrow versus Judge Dredd]

  • mollishka says:

    Your definition of "commmon sense," while true, is a bit of an oxymoron... it's fairly uncommon for people to actually think instead of, you know, not.
    The biggest difference between real world science and high school science (even in the sense of when high schoolers are taught, here is how scientific research is done) is the idea that not all experiments start with a well-defined hypothesis, which you mentioned. But even when an experiment is motivated by "let's test this hypothesis," if the data don't support the hypothesis, then that doesn't mean it's a "failed" experiment ... it just means that your hypothesis/theory made some invalid assumption that you have to now reconsider. (Experiments never fail; they just run out of funding?)
    Oh, and I just noticed you added my blog to your blogroll... thanks for the link!

  • Anonymous says:

    Well, I don't agree with much in this description except the last part about its iterative nature. (And with Vos Post, its social nature.)
    If the success of science was only due to it being organized common sense, it would have been successful much earlier in history, I think. The difference seems to be that quantitative testing (and consequently, prediction) eventually become adopted, instead of merely observation and description. Or in other words, that science effectively rejects the false.
    Whether science approaches 'Truth' or reality is mostly a philosophical dilemma. (But I guess more scientists are realists than metaphysicists.) But the definition of fact is not.
    "Fact" could be more than a description of our observations. It could also be a description of the existence of the processes we observe, and of the laws that we extract. I would say (with several biologists, I think) that evolution in this sense is a fact of nature, and so is energy conservation.

  • Enigman says:

    Cf. van Benthem: "the distinction between scientific method and common sense - once thought so iron-clad - is evaporating. There is just one form of rationality, whether displayed in the kitchen or the Halls of Minerva", from: http://www.illc.uva.nl/Publications/ResearchReports/PP-2005-21.text.pdf

  • David Harmon says:

    "The scientific method is nothing more than a system of rules to keep us from lying to each other."
    Also to keep us from lying to ourselves!

  • David Ng says:

    That's awesome! Kind of like an advanced version to my beginner slide (http://www.boingboing.net/2010/08/23/where-are-all-the-aw.html)