Falsificationism is false

Why a popular idea in the philosophy of science shouldn't be so popular.

Recently, I was on Spencer Case’s Micro-Digressions podcast to talk with him and with MIT’s Alex Byrne about the “In Defense of Merit in Science” paper that got some press attention a few months ago. It’s a nice conversation; you should check it out. (There’s also a “part 2” where I get into it with Byrne on the meaning of gender terms; that will come out in a few weeks and I’ll post about it here when it does.) I want to take a bit of time here to expand on one of the ideas that I threw out in that first podcast episode, and perhaps I’ll come back to this well later. But today: Karl Popper, and why he’s not good.

I mean, of course Popper is good. He’s one of the great philosophers of science, his work is incredibly influential, and it has every right to be influential. His 1959 book The Logic of Scientific Discovery (itself an English “rewriting” of a 1935 German original) is a landmark work.¹ Yet today few philosophers would endorse his views. This is something of a weird position to be in, because while philosophers of science are lukewarm on Popper, scientists themselves can’t get enough of the guy.

No scientist is entirely without a philosophy of science. Every scientist has some answer they can give to basic questions like “What is science?” and “Why is science good (and metaphysical speculation bad)?” But of course those are philosophical questions; you can’t figure out the answers to them by running any kind of empirical test. So even though many scientists eschew philosophy of science, they have philosophical views about science. And for most scientists, their philosophy of science is just Popper’s philosophy of science. Why? Hard to say; it just seems to be in the water in science buildings, little nuggets of wisdom and tradition that are passed on in beer sessions after the lab closes for the night and similar kinds of informal training. But it’s not because Popper’s views are correct.

The bad Popper view that I want to focus on today is falsificationism. This is the idea that a claim is “scientific” if and only if it is falsifiable (by empirical tests). That is:

Falsificationism: A claim is scientific if and only if it is falsifiable. A claim is falsifiable if and only if it logically entails the truth of some claim that we can empirically determine to be false. (If that empirical claim is false, then it follows that the scientific claim that entails it is also false.)

Before we dive into the technical issues, note at the offset that this is at least a non-obvious answer to the question of what makes a claim “scientific.” Why say that claims are scientific when they’re falsifiable rather than say that they are scientific when they’re verifiable? Verifiability seems the obvious candidate; we develop a theory and then test to see if it’s true, not to see if it’s false, no? When scientists ask whether or not a claim is falsifiable, that’s a rather unnatural question to ask. It’s the sort of question that would only be formulated under the influence of a certain theory — Popper’s theory, in this case.

So what is Popper’s theory? Popper focuses on falsification because he thinks that verification is impossible. No amount of data could provide even an iota of support that some general theory is true. His reasons for thinking this are essentially Hume’s reasons: claims about how the past have been provide no guarantee about how the future will be or about how things are in general, past, present and future. The standard response to Humean worries is to say that general theories are not conclusively proved by verifying evidence, but are made more probable by verifying evidence. But this relies on a kind of “inductive logic of probabilistic confirmation,” and Popper thinks that this kind of logic is incoherent. (Karl Popper: not a Bayesian.) So he relies instead on falsification; logically, falsificationism requires only deductive logic rather than inductive logic, and Popper thought that deductive logic is much more secure than inductive logic.

What are the problems with falsificationism? Here’s Lee Jussim (in the comments at his own blog, responding to the criticisms we made on the podcast):

It is true that my view, and I think most of my co-authors though I am not sure, is that to be taken as serious science, a claim or theory should generate *something* falsifiable. Philosophers tie themselves up in knots torturing the obvious, imho, at last sometimes. I have seen edge cases presented as if they debunk falsificationism, such as predictions that cannot be falsified yet most would consider science, such as, there is life somewhere in the universe or that global warming will render half the Earth uninhabitable in 200 years. And yet, both generate falsifiable predictions, or, at least, can. Is there life on Mars? Titan? Was Oumawauma a derelict alien artifact? If one has a model that predicts warming is so severe 200 years out, then it should be *even better* at predicting warming 5 or 10 or 20 years out. All of these are falsifiable predictions.

With respect: no, that’s not the problem with falsificationism; the objections are deeper than problems with “edge cases.” (And pointing out these problems isn’t “torturing the obvious!”²) And look at Jussim’s examples: all of these claims, save one, are falsifiable. Whether there is life on Mars or Titan is certainly falsifiable: just go there and comb every inch of the planet to look for life, and if there’s no life anywhere there, you’ve falsified the hypothesis. (There might be huge practical issues with checking these things, but there is no bar to falsifying them in principle. They’re falsifiable because they entail claims which we could empirically discover to be false) Whether global warming will render half of the Earth uninhabitable in 200 years is also falsifiable in principle; just wait and find out. These are, I think, relatively uncontroversial verdicts (but for the further concerns I’ll raise below).

The claim that there is life somewhere in the universe (assuming an infinitely large universe), on the other hand, is not falsifiable. It’s famously not; this is a textbook example! Now, it is verifiable. There are some observations you can make that would conclusively prove that it is true: Find life on an alien planet, and the theory is verified. But there’s no (finite) amount of data that could ever possibly falsify that theory. If there are an infinite number of planets in the universe, you can visit as many as you like but, even if you find life on none of them, there’s always another planet to explore and maybe life exists on that next planet. You cannot derive any falsifiable predictions from the hypothesis that life exists somewhere. (You can derive the claim that life exists somewhere from the claim that life exists on Mars, but you can’t derive the claim that life exists on Mars from the claim that life exists somewhere.) Claims to the effect that something will be found somewhere are never falsifiable, because no matter how much you seek and don’t find, it might always be in the next place you look. This is what people mean when they say “You can’t prove a negative.” And yet the claim that life exists somewhere in the universe is a scientific claim. So falsification gets the wrong results here; this is a genuine, textbook counter-example.

I guess Jussim might concede that this is a real counter-example, but that falsificationism works most of the time, and that’s good enough. I’d respond by saying that counter-examples show the theory is false, and false theories aren’t “good enough” if we want real understanding here. And besides, there are other problems with falsificationism.

One is the problem of arbitrary conjunctions. Take some respectable scientific claim, like the claim that the Earth is roughly spherical (or whatever), and then take some perfectly absurd bit of metaphysical nonsense, like the claim that the Absolute is Perfect. That the Earth is roughly spherical is falsifiable; it has entailments that could be shown to be false with the right empirical data. And that the Absolute is Perfect is not falsifiable; it has no such entailments. So far, so good. But now consider the conjunction of those two claims: The earth is roughly spherical and the Absolute is Perfect. This conjunction is false if either conjunct is false. That means that it can be falsified by falsifying either conjunct. The claim that the earth is roughly spherical is falsifiable. So the claim that the earth is roughly spherical and the Absolute is Perfect is also falsifiable. So that claim is scientific. That’s a bad result! We shouldn’t be able to make a claim scientifically respectable just by conjoining it to a scientifically respectable claim. Falsificationism entails that we can.

You might think you see an easy fix for this: Just don’t allow for conjunctions to count as falsifiable or non-falsifiable. Simple claims can be tested to see if they’re falsifiable, and complex claims are scientific only if they’re made up of falsifiable, scientific claims. But this suggestion runs into the final and perhaps most devastating problem for falsificationism: the problem of holism.

The problem of holism begins from the observation that particular claims can never be tested in isolation. Testing a claim always relies on a host of “background assumptions” or “auxiliary hypotheses.” The easiest way to grasp the idea of a “background assumption” is to think of the assumption that my lab equipment is working correctly. My theory that I’m testing in the lab only generates falsifiable predictions if my lab equipment for testing the theory is working correctly. Without that assumption, my theory has no empirical implications. More generally, any theory that says how things are in the world must incorporate additional assumptions about how the world links up with our experiences in order for that theory to have implications for our experiences. So we never test particular theories; we always test collections of theories, as a whole: the theory itself that we’re interested in, together with background assumptions that link the theory to observation. So individual theories are not falsifiable. This makes falsificationism a perfectly terrible test for whether or not an individual theory is scientific.

WVO Quine even argued (more controversially) that, since background assumptions also implicate other theories (e.g. the scientific theories that went into constructing the lab equipment), taking holism seriously means that it’s impossible to test even relatively small collections of theories together: our total body of theory faces the tribunal of our total body of experience all at once. There’s no way to slice our body of theories up, at all, and thus no way to slice them up into scientific and non-scientific parts.

Even if Quine’s stronger view is wrong, the fact that we can only test a theory relative to a body of background assumptions makes trouble for Popper in two ways. First, the obvious way to solve this problem (focus on larger collections of theories) pulls in the opposite direction as the obvious way to solve the problem of arbitrary conjunctions (focus on smaller bits of theory). So solving both of these problems together is a real challenge. Second, it shows that we can never really falsify a theory; we only falsify the conjunction of the theory with some background assumptions. And if we have a choice of abandoning the theory or the background assumptions in light of falsifying data, we can always just reject the background assumptions to save the theory. Because theories are always falsifiable only with the aid of background assumptions, falsifying data need never impugn the theory in question; we can always blame our lab equipment (or some other, related bit of theory) when the experiment turns out wrong.

These problems were well-understood in Popper’s time.³ Popper had a response prepared, which is — basically — to abandon the whole idea of falsification as a way of separating out scientific from pseudo-scientific claims. His new idea is to think of falsification as a methodology, where methodology, for Popper, is less a set of steps and more a kind of temperament. Methodological falsificationism is just taking a hard-headed attitude towards your own theories. If your experiment doesn’t work out, and you can either blame your theory or blame your lab equipment, a scientifically-minded person doesn’t blame their lab equipment. They take it on the chin, reject their theory, and move onto the next one.

It’s worth emphasizing that this is a completely different view. Up until now, I’ve been referring to a certain view as falsificationism, which states what it takes for a claim to be falsifiable, and further states that claims are scientific if and only if they’re falsifiable. Let’s call that demarcation-falsificationism, since it’s Popper’s solution to his “demarcation problem” of distinguishing science from non-science. This new view is not a demarcation criterion. It’s more like a bit of advice to scientists about how to comport themselves if they’re acting like a true scientist. Failures of falsification-as-methodology are failures not of theories but of people. The essential criticism is “stop blaming your lab equipment so much and let go of your theory,” not “that theory makes no falsifiable predictions.” Let’s call this latter view methodological-falsificationism.

So, dialectically, what’s happened is that Popper put forward a theory for classifying claims as scientific and non-scientific, it was pointed out by many people that his theory sucked, and so Popper abandoned it (?)⁴ for a different theory that he used the same word for (“falsificationism”). But scientists have since followed Popper in using his term “falsificationism” to endorse the theory that Popper himself abandoned(?) because it sucked, and they’ve carried on this practice for nearly a century even though philosophers have been shouting the problems with this theory since the 1940s! This is weird and bad!

Even methodological-falsificationism is highly controversial. The basic problem is that sometimes your lab equipment breaks and you should blame it. The more sophisticated version of this problem comes from Thomas Kuhn. Kuhn argued that background assumptions don’t come from nowhere. They’re hammered out progressively as we work out a theory, as a way to bring our theory into better alignment with the observed data. Sometimes things get to a point where we can’t see a way to bring our theory into alignment with the data and we start to worry that the theory was on the wrong track to begin with, whatever initial success it might have enjoyed; this is “revolutionary science.” But the vast majority of science is “normal science,” which is characterized by a practice of ruthlessly refining background assumptions while leaving the hard core of the theory untouched. (The Kuhn-inspired Imre Lakatos would call these background assumptions the “protective belt” of the theory that insulate it from empirical refutation.)

Popper did not deny that we often blame background assumptions when our experiments don’t work out, he just thought this was a bad thing. Kuhn, for his part, argued that this is necessary to good science. We can only really figure out whether a theory works by working on it and refining it. Copernicus’s geocentric theory was a mess in its first incarnation. It was only after centuries of modification to the protective belt that the geocentric hard core was able to be part of a general theoretical framework that surpassed the old Ptolemaic framework. Popper’s methodology of abandoning a theory as soon as it hits trouble is a recipe for theoretical breadth; you’re always moving on to the next thing. Kuhn’s normal science is a recipe for theoretical depth; you really explore the idea to figure out how well it can be made to work. Both are important for science.

In Popper’s defense, one might say that modifying the “protective belt” really is a way of pursuing falsificationist methodology. After all, that doesn’t just mean blaming your lab equipment, it can also mean modifying other substantial bits of theory. To modify a theory is to reject one version of a theory and replace it with a better version of the theory. So we still have rejection and replacement. All that’s left is negotiation over how “central” the pieces of the theory we should be rejecting and replacing need to be, and this should be done on a case-by-case basis, relying on expert judgment in light of the available data. Perhaps this can reconcile methodological-falsificationism with Kuhn’s insights. But this way of thinking concedes the basic point of the holist critique and thus gives up on demarcation-falsificationism. If we are counting modifications to the protective belt as “rejecting” the theory, that’s because we’re considering the theory and its background assumptions to be one unified whole. So the debate over methodological-falsificationism ends up reinforcing one of the main problems with demarcation-falsificationism.

Methodological-falsificationism is defensible, although controversial. Demarcation-falsificationism has such profound problems that not even Popper himself really accepted it. Yet scientists (almost) universally accept demarcation-falsificationism, and are (apparently) entirely ignorant of the problems with it. That’s not great!

I teach this all in the first couple weeks of my Intro to Philosophy of Science class. If a student gets to the first midterm and can’t recite some version of what I’ve just said above, they’re not going to get a good grade. It’s absolutely embarrassing that many working scientists don’t understand this stuff. (You don’t even need to take a full seminar! Just sit in for the first couple weeks, or read, like, a quarter of an intro text!)⁵ It’s not like scientists don’t care about philosophy of science; every scientist has a philosophy of science. And it’s not like scientists aren’t getting trained in philosophy of science; the widespread use of the term “falsification” is abundant evidence of at least informal training in the philosophy of science. The problem is that scientists train themselves very badly in the philosophy of science because they’re just repeating the half-remembered, half-understood bits of Popper that they picked up from their PhD advisor in the bar after the lab closed. The most embarrassing thing about this situation is that more scientists aren’t embarrassed by it.

And I guess this is my core objection to the “In Defense of Merit” paper.⁶ It’s structured like the Declaration of Independence: a bold statement of high principle, followed by a list of grievances. That list of grievances is, to my eye, extremely compelling by itself. The paper is worthwhile for that part alone. But the bold statement of high principle is pure philosophy of science, being written by 28 distinguished scientists,⁷ none of which seem to have even a 101-level understanding of the subject on which they’re writing. Say what you will about Thomas Jefferson cribbing from John Locke in the Declaration, at least he read Locke. I’ll borrow this last point from Spencer Case: Either you care about philosophy of science, or you don’t. If you don’t care about philosophy of science, why dedicate the first 1/3 of your paper to it? (Shut up and calculate!) If you do care about it, then why aren’t you taking it seriously enough to learn the basics?

1. I’ve found this makes situating Popper’s views in time rather interesting. Since the 1959 version of the book is what is commonly-cited, he’s often seen as a post-positivist philosopher of science, yet the original work was done during the heyday of logical positivism. The 1935 book is a “rewriting” rather than a direct translation, and I (not speaking German) wonder how much changed from the German to the English edition. Perhaps some of the inconsistencies that are so frustrating about Popper stem from the fact that his central work is really of two times, separated by 24 years that contained huge events in philosophy, science, and world events. (Popper, Jewish, lost a lot of family in WW2 and was profoundly changed by the experience.)

2. This reads a bit like the point you get to 1/3 of the way through any early Socratic dialogue, where Socrates’s opponent says “Ah, ok, I see, you’ve raised a problem for my view as stated, but it doesn’t seem serious. I’m sure there’s an easy fix to be made.” And Socrates’s eyes start to gleam. “I’m sure you’re right, and I’m happy to hear it. What fix do you propose?”

3. At least, they were well-understood by 1959. I’m not sure about the 1935 German version, but these problems were well-worked-over, by Carl Hempel in particular, in the 1940s.

4. It’s not clear if Popper ever abandoned demarcation falsificationism; perhaps his simultaneous endorsement of demarcation-falsificationism and methodological-falsificationism in the 1959 book is a product of how his thinking evolved over time. He apparently endorses both views, but he motivates methodological-falsificationism by pointing out the problems with demarcation-falsificationism. It’s a hard to know how to interpret that.

5. Ok, ok, I only get to Kuhn later. But “Who is Popper, and what is falsificationism?” is Week 3, and it’s in the first quarter of the textbook I use.

6. To be clear, the “In Defense of Merit” paper only mentions falsificationism in passing, but it invokes many other concepts that are equally contentious, and for equally good reason. I’m focusing on falsificationism here because scientists blithely talking about falsifiability has bugged me for a long time. I might come back to the topic later to pick at other points that bugged me. Again, check out the podcast discussion!

7. And also Peter Boghossian. Boghossian is a philosopher, but with no background in philosophy of science that I’m aware of.