It has become common for pundits and other sundry political commentators these days to talk about their “priors.” As in, “that confirms my priors” or “my prior is that…” It’s everywhere. It’s the new lingo. But what, you might ask, is a “prior?” What are these people talking about? The answer is simple. “Prior” means “belief.” That’s all. Whenever anyone talks about their priors, they’re just talking about what they believe or what they think. “My prior is that…” means “I believe that…” “That confirms my priors” means “That confirms my beliefs.” That’s all.
Now, I’m sure people will object that this oversimplifies. This does not oversimplify; it simplifies precisely enough. But yes, there is a more complicated story to tell here. Let’s talk about Bayesianism.
Bayesianism is the hot new thing in statistics. It’s been around for a while (centuries, in fact), but it’s recently gotten very popular. Bayesianism is named after the Reverand Thomas Bayes, an 18th century mathematician who came up with Bayes’ Theorem: Pr(H|E) = Pr(E|H)*Pr(H)/Pr(E). Lots of people think that this is a very important equation. But what does it say? Let’s break it down a bit.
Bayes’ Theorem is a theorem about probability. Pr(X) just means “the probability of X.” So Pr(H) and Pr(E) in Bayes’ Theorem refer to the probabilities of E and H. E and H are just variables - they can stand for any propositions. But when you see “E” think of “evidence,” and H is “hypothesis.” So we’re dealing with the probabilities of evidence and hypotheses. Bayes’ Theorem can apply to any propositions, but people get excited about Bayes’ Theorem when they apply it to the case of evidence for a hypothesis. Another bit of notation you need to know is Pr(X|Y). That is the notation for a conditional probability. Pr(X|Y) is read as “the probability of X conditional on Y” or “the probability of X, given Y.” So Bayes’ Theorem says that:
The probability of a hypothesis, given some evidence, is equal to the probability of that evidence, given the hypothesis, times the probability of the hypothesis, divided by the probability of the evidence.
Pr(H|E) = Pr(E|H) * Pr(H) / Pr(E).
One thing to note about this is that it is a theorem. It is trivial. It follows from the definition of Pr(X|Y). Pr(X|Y) is the probability of X given Y, but it is typically defined as Pr(X&Y)/Pr(Y). (This is the “ratio definition of conditional probability.”) Bayes just showed that his famous theorem follows from the ratio definition of conditional probability.
Pr(H|E) = Pr(E|H) * Pr(H) / Pr(E)
Pr(H&E)/Pr(E) = Pr(H&E)/Pr(H) * Pr(H) / Pr(E)
Just substitute the ratio definition in for each instance of conditional probability and divide through to prove that these are equivalent. The theorem follows easily from the definition given basic algebra. This is high school math.
So why do we care? What makes this such an important theorem? Well, this matters because the thing that we care about when we’re collecting evidence is Pr(H|E). We want to know how likely our hypothesis is, given the evidence we’ve collected. But standard statistical tests, like a p-value test, only give Pr(E|H). They say how likely it is for you to get a piece of evidence on the assumption that they hypothesis is true. So we don’t have what we want. Bummer.
But Bayes’ Theorem promises to turn what we can get into what we want through the application of a little bit of high school math. To get what we want, the probability of H given E, we just need to start with the probability of E given H, then multiply it by the probability of the hypothesis and divide by the probability of the evidence. All of these bits of the formula have been given names. What we want - the probability of H given H - is known as the posterior probability of H. The probability of E given H is known as the likelihood of E given H. And Pr(E) and Pr(H) are the prior probabilities of E and H. Or “priors” for short.
Now a thought may have already occurred to you. In order to figure out the probability of H given E, we need three things: the likelihood, and the prior probability of E and H. Focus on that list thing: We need to know the probability of the hypothesis H. But… isn’t that exactly what we’re trying to figure out in the first place? What we’re trying to figure out is the probability of H given this piece of evidence. But knowing that requires us to know the probability of H given all evidence before this piece of evidence. That’s what the prior is. So how can we know the prior? That’s the same kind of problem as the one we started with.
Now this is an exceedingly difficult problem - the “problem of the priors” - and some people think it renders the entire Bayesian approach to epistemology basically useless. But there’s a very common way to answer this worry.
Take a step back: what is a probability to begin with? There are lots of wildly different, conflicting answers to that question. I won’t take you through all the possibilities; that’s a huge topic for a different blog post. But one popular answer to that question is subjectivism. According to subjectivism, a probability is just a degree of belief. There are some claims that you are very confident in; like the claim that 2+2=4. We might say that you are 100% confident that that claim is true. And there are some claims that you are very doubtful of; like the claim that 2+2=fish. We might say that you are 0% confident that that claim is true. Other claims are ones you might have a middling confidence in. How confident are you that Joe Biden will win reelection in 2024? 50%ish? A bit higher? A bit lower? Whatever it is, the subjectivist account of probability says that your level of confidence or degree of belief is your own personal probability that the claim is true.
So, what’s the prior probability of H? For the subjectivist, there is no one thing which is the prior probability of H. There’s just your prior probability of H. And your prior probability of H is just however confident you are that H is true. Your “prior” is just your degree of belief before the new evidence comes in. Knowing your priors and the likelihood, you can then calculate your posterior probability in H given E. Then, if you learn that E is true, subjective Bayesians say that you should update by conditionalization, which means that your degree of belief should change from Pr(H) to Pr(H|E).
So, in short: your “prior” is just your degree of belief. Knowing your priors and the likelihood, you can calculate Pr(H|E) using a bit of high school math. Then, if you learn E, you should update your priors to what you calculated Pr(H|E) to be.
If this sounds a bit silly, that’s because it is.
But because this way of thinking has become very popular (due largely, as far as I can tell, to the influence of Nate Silver), when some people talk about what they think, they don’t say what they “think” or “believe.” They talk about their “priors.”
So “priors” is just a way of saying “beliefs” that assumes a whole bunch of highly substantive positions about the nature of probability and how you ought to respond to evidence. Those substantive positions are very controversial. They’re in vogue these days, but there’s lots of problems with those positions. I’m not a subjective Bayesian. Now if you’re a subjective Bayesian, go ahead and talk about your priors and keep on pretending that you’re doing math every time you get new evidence. But for the rest of us? Please, for the love of God, just say “belief.”
The main reason people should stop saying, "That confirms my priors," is because it doesn't even make any sense. Any evidence you get at all should change your prior probability, unless you've gotten pieces of evidence that weigh both for and against the hypothesis and perfectly cancel each other out, in which case the net effect is no evidence. So it's by definition impossible to "confirm your priors": If your prior stayed the same, then you got no net evidence either way, and if you got evidence that confirms something, you should necessarily update your priors.
As you say in the article, what people really mean by "confirm my priors" is "confirm my beliefs." That can be done by simply finding evidence that weighs in favor of the hypotheses you already thought were more likely. This will make you more confident in your beliefs, but that's not the same thing as being more confident in your priors. By definition, becoming more confident in your beliefs means that you no longer agree with your old prior. People are just using the word "prior" wrong - to represent a proposition that you hold a binary attitude of belief or nonbelief towards, rather than to describe a degree of confidence in a proposition.
As a side note, I think the SMBC comic is wrong. The entire point of Bayesianism as a philosophy is to make you less confident in your beliefs by treating them as non-binary judgements that change over time, rather than just thinking in terms of "this claim is true/false," which tends to make people overconfident. And the Bayesians that I know of, Nate Silver among them, actually succeed at this and tend not to render overconfident judgements. They also don't pretend to use math unless they actually are doing the math. Meanwhile, other pundits tend to produce very overconfident judgements. To be fair, though, I might have a biased sample that includes a lot of Bayesians who do it right, rather than the ones who misuse Bayesian reasoning.
Good article overall, but the use of priors and Bayesian reasoning is downstream of the rationalism community rather than Nate Silver