Conditional Probability

Conditional probability answers one question: once you learn that some event happened, how should the probability of another event change? It is how you fold fresh information into an old belief. Almost every question about reacting to news, a signal, or a test result is, underneath, asking you to compute one.

What "given" means

The notation $P(A \mid B)$ reads "the probability of $A$ given $B$." You are told that $B$ definitely occurred, and you ask how likely $A$ is inside that smaller world. Learning $B$ throws away every outcome where $B$ is false and leaves only the outcomes where $B$ is true. The question becomes: of that shrunken world, what fraction also has $A$?

The formula

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)} .$$

The symbols: $P(A \cap B)$ is the probability that $A$ and $B$ both happen (the symbol $\cap$ means "and"); $P(B)$ is the probability of $B$ on its own; and we require $P(B) > 0$, since you cannot condition on something that never occurs. In words: out of all the ways $B$ can happen, what share also has $A$? We divide by $P(B)$ to renormalize, because $B$ is now the entire universe, so its probability has to climb back up to 1.

Multiply both sides by $P(B)$ and you get the equally useful multiplication rule:

$$P(A \cap B) = P(B)\,P(A \mid B) .$$

Independence is the special case where nothing changes

Sometimes learning $B$ tells you nothing about $A$. Then $P(A \mid B) = P(A)$, and the multiplication rule collapses to $P(A \cap B) = P(A)\,P(B)$. That is exactly what independence means: the events do not move each other. Any time you multiply two probabilities together as if they were unrelated, you are quietly assuming independence, and if it is false your answer is wrong.

Worked example

Roll one fair die. Let $B$ be "the roll is even" and $A$ be "the roll is at least 4." You are told the roll came up even; how likely is it to be at least 4?

• $B = \{2,4,6\}$, so $P(B) = 3/6 = 1/2$.
• $A \cap B$ is "even and $\ge 4$" $= \{4,6\}$, so $P(A \cap B) = 2/6 = 1/3$.

Plug in:

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)} = \frac{2/6}{3/6} = \frac{2}{3} .$$

So among the even rolls, two of the three ($4$ and $6$) clear the bar, giving $2/3$. Notice the unconditional answer was different: $P(A) = P(\{4,5,6\}) = 1/2$. Learning the roll was even pushed the probability up from $1/2$ to $2/3$, which is the whole point of conditioning.

Common pitfalls

• Flipping the bar. $P(A \mid B)$ is not $P(B \mid A)$. "The probability a test is positive given you are sick" is very different from "the probability you are sick given a positive test." Swapping them is the classic prosecutor's fallacy, and correcting it is exactly what Bayes' theorem is for.
• Forgetting to renormalize. The division by $P(B)$ is not optional. Skip it and your "probabilities" inside the conditioned world will not add up to 1.
• Assuming independence for free. Multiplying $P(A)P(B)$ is only valid when the events truly do not affect each other. Correlated trades, correlated defaults, and repeated draws without replacement all violate it.
• Ignoring the base rate. A conditional probability can be small even when the evidence looks strong, because a rare event stays rare. To combine the evidence with the base rate correctly you need the The Law of Total Probability in the denominator.

Conditional probability, the The Law of Total Probability, and Bayes' theorem are three views of the same identity. Master this one and the other two are a line of algebra away. For the formal ground it all sits on, see Probability Spaces.