Conditional probability
Before reading this guide, it is highly recommended that you read Guide: Introduction to probability.
| Narration of study guide: |
Introduction
Conditional probability is a fundamental concept in probability theory. It describes how the likelihood of an event changes when additional information becomes available. In practice, it is used to update probabilities based on the outcome of related events and is widely applied in areas such as data analysis, forecasting, and decision-making.
It also forms the basis for several key results in statistics, including the multiplication rule, the law of total probability, and Bayes’ theorem. This guide covers the definition of conditional probability, methods for calculating it, ways to test for independence, and highlights the distinction between independent and disjoint events.
Conditional probability
In everyday life, the likelihood of an event often changes as new information becomes available. For example, if you’re planning to leave the house and notice dark clouds gathering, you might assume it’s more likely to rain and decide to bring an umbrella. This shift in expectation can be understood using conditional probability.
Conditional probability is the probability of an event occurring given that another event has already occurred. It quantifies how one event influences the likelihood of another.
Suppose that \(A,B\) are events in a sample space, where \(\mathbb{P}(A),\mathbb{P}(B)\) are the probabilities that \(A\) and \(B\) occur.
You can write
- \(\mathbb{P}(A\cap B)\) (read ‘\(\mathbb{P}\) of \(A\) and \(B\)’) to mean the probability of both \(A\) and \(B\) happening.
- \(\mathbb{P}(A\mid B)\) (read ‘\(\mathbb{P}\) of \(A\) given \(B\)’) to mean the probability of \(A\) happening given that \(B\) has already happened,
Then
\[ \mathbb{P}(A \mid B)=\dfrac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}. \]
is the conditional probability that \(A\) happens given that \(B\) has already happened.
For a more detailed explanation as to where \(\cap\) comes from as notation, see [Guide: Operations on sets].
Conditional probability is only defined when \(\mathbb{P}(B) > 0\), meaning that the event \(B\) must have a chance of occurring. This is because it doesn’t make sense to ask about the probability of \(A\) given \(B\) if \(B\) has no chance of occurring. You can see this reflected in the formula for conditional probability since dividing by zero would make \(\mathbb{P}(A \mid B)\) undefined.
Here’s an example of conditional probability. You can remember from Guide: Introduction to probability that rolling a fair six-sided die consist of independent events.
Suppose you roll a fair six-sided die. The sample space of possible outcomes is \(\{1, 2, 3, 4, 5, 6\}\). You are told that the number \(X\) is greater than \(2\), so what is the probability that the number is even?
Knowing that the result is greater than \(2\), the possible outcomes are now \(\{3, 4, 5, 6\}\) and among these outcomes, the even numbers are \(4\) and \(6\). Based on a glance of the possible outcomes, it follows that the probability of an even number given that the outcome is greater than \(2\) should be \(2/4\).
You can use the formula for conditional probability to check that this intuition is true. Here, the probability of rolling an even number given that the roll is greater than two is:
\[ \mathbb{P}(\textsf{Even} \mid X>2) = \dfrac{\mathbb{P}(\textsf{Even} \cap X > 2)}{\mathbb{P}(X > 2)}. \]
You can find the individual probabilities in the fraction to work this out.
- \(\mathbb{P}(\textsf{Even and } X > 2)\) is the probability of rolling an even number above \(2\), meaning either a \(4\) or a \(6\) out of a possible outcome set of \(\{1,2,3,4,5,6\}\). So:
\[ \mathbb{P}(\textsf{Even} \cap X>2) = \dfrac{2}{6}. \]
- \(\mathbb{P}(X > 2)\) is the probability that the roll is greater than \(2\), which includes the outcomes \(3, 4, 5,\) and \(6\) out of a sample space of \(\{1,2,3,4,5,6\}\), so:
\[ \mathbb{P}(X>2) = \dfrac{4}{6} \] Substituting these results into the formula gives:
\[ \mathbb{P}(\textsf{Even} \mid X>2) = \dfrac{2/6}{4/6} = \dfrac{2}{4} = \dfrac{1}{2} \] which confirms your previous intuition.
Here’s another example, this time with drawing cards.
Now suppose you draw a card at random from a standard deck of 52 cards. You are told that it is a face card: that is, a Jack, Queen, or King. What is the probability that the card is a King?
Since you know the card is a face card, the possible outcomes are now limited to the four Jacks, four Queens, and four Kings. There are twelve in total, four of which are Kings, and so you might expect the probability of a King given a face card to be \(4/12 = 1/3\).
You can check your intuition using the conditional probability formula, which in this case is:
\[ \mathbb{P}(\textsf{King} \mid \textsf{Face card}) = \dfrac{\mathbb{P}(\textsf{King} \cap \textsf{Face card})}{\mathbb{P}(\textsf{Face card})}. \] First, you can find the individual probabilities:
- \(\mathbb{P}(\textsf{King and Face card})\) is the probability that the card is a King, since every King is also a face card. Since there are \(4\) Kings in the deck:
\[ \mathbb{P}(\textsf{King} \cap \textsf{Face card})= \dfrac{4}{52}. \]
- \(\mathbb{P}(\textsf{Face card})\) is the probability that the card is any face card. There are \(12\) face cards in total, so:
\[ \mathbb{P}(\textsf{Face card}) = \dfrac{12}{52}. \]
Substituting these into the conditional probability formula gives:
\[ \mathbb{P}(\textsf{King} \mid \textsf{Face card}) = \dfrac{4/52}{12/52} = \dfrac{4}{12} = \dfrac{1}{3}. \]
So, given that the card is a face card, the probability that it is a King is \(1/3\) - which is the same as your intuition above.
Multiplication rule
Sometimes you want to find the probability that two events both occur. For example, suppose you draw two cards from a deck, one after the other. You might know the probability that the first card is a Queen, and you might also know the probability that the second card is a Queen given that the first card was a Queen. What if you want to find the probability that both cards are Queens?
The multiplication rule allows you to calculate the probability that two events both occur, using a conditional probability and the probability of one of the events. It is especially useful when you have partial information about two related events, or when the outcome of the second event depends on the outcome of the first event.
If \(A\) and \(B\) are events, then the probability that both occur, \(\mathbb{P}(A \cap B)\), is given by:
\[ \mathbb{P}(A \cap B) = \mathbb{P}(B) \mathbb{P}(A \mid B). \]
Because the probability of \(A\) and \(B\) occurring is the same as the probability of \(B\) and \(A\) occurring, the multiplication rule also works the other way around:
\[ \mathbb{P}(A \cap B) = \mathbb{P}(B\cap A) = \mathbb{P}(A) \mathbb{P}(B \mid A) \]
Both forms are correct, so you can choose the one that matches the information you have.
Let’s look at an example.
Suppose you draw two cards from a standard deck of cards, one after the other, without replacement. What is the probability that both cards drawn are queens?
To solve this, you can use the multiplication rule: \[ \mathbb{P}(\textsf{First queen} \cap \textsf{Second queen}) = \mathbb{P}(\textsf{First queen}) \mathbb{P}(\textsf{Second queen} \mid \textsf{First queen}). \] First, calculate the probability that the first card is a queen and the probability that the second card is a queen given the first was a queen. There are \(4\) queens in a deck of \(52\) cards, so: \[ \mathbb{P}(\textsf{First queen}) = \dfrac{4}{52}. \] If the first card was a queen, there are now \(3\) queens remaining in a \(51\) card deck: \[ \mathbb{P}(\textsf{Second queen} \mid \textsf{First queen}) = \dfrac{3}{51} \] Substituting these into the multiplication rule gives: \[ \mathbb{P}(\textsf{Two queens})=\left(\dfrac{4}{52}\right)\left(\dfrac{3}{51}\right) = \dfrac{12}{2652}=\dfrac{1}{221} \approx 0.00452. \] So, the probability of drawing two queens without replacement is approximately \(0.00452\).
Checking for independence using conditional probability
Conditional probability can also be used to check whether two events are independent. If two events \(A\) and \(B\) are independent, then knowing that \(B\) has occurred does not change the probability of \(A\) occurring. In other words:
\[ \mathbb{P}(A \mid B) = \mathbb{P}(A). \] This is equivalent to saying that
\[ \mathbb{P}(A \cap B) = \mathbb{P}(A) \mathbb{P}(B) \]
If either of these conditions do not hold, then the probability of \(A\) is affected by the occurrence of \(B\), which means \(A\) and \(B\) are dependent events.
For more about independent and dependent events, see Guide: Introduction to probability.
Imagine you roll a fair six-sided die as in Example 1. Let \(A\) be the event that the die shows an even number, and let \(B\) be the event the die shows a number greater than \(4\).
First, find:
- \(\mathbb{P}(A)\), the probability the die shows an even number (\(2, 4,\) or \(6\)):
\[ \mathbb{P}(A) = \dfrac{3}{6} = \dfrac{1}{2}. \]
- Now find \(\mathbb{P}(A \mid B)\), the probability the number is even given that it is greater than \(4\). The outcomes greater than \(4\) are \(5\) and \(6\) and among the two, only \(6\) is even. So:
\[ \mathbb{P}(A \mid B) = \dfrac{1}{2} \]
Since:
\[ \mathbb{P}(A \mid B) = \dfrac{1}{2} = \mathbb{P}(A) \]
the events \(A\) and \(B\) are independent.
In Example 2, you worked out \[ \mathbb{P}(\textsf{King} \mid \textsf{Face card}) = \dfrac{4/52}{12/52} = \dfrac{4}{12} = \dfrac{1}{3}. \] Since the probability \(\mathbb{P}(\textsf{King}) = 1/13\), it follows that \[\mathbb{P}(\textsf{King} \mid \textsf{Face card}) = \frac{1}{3}\neq \frac{1}{13} = \mathbb{P}(\textsf{King})\] and so drawing a King and drawing a face card are dependent events.
Suppose you draw a card at random from a standard deck. Let \(A\) be the event that the card is a Queen, and let \(B\) be the event that the card is a heart.
First, find the individual probabilities:
- \(\mathbb{P}(A)\) is the probability the card is a Queen. As there are \(4\) Queens in the deck:
\[ \mathbb{P}(A) = \dfrac{4}{52} = \dfrac{1}{13}. \]
- \(\mathbb{P}(B)\) is the probability the card is a heart. As there are \(13\) hearts in total:
\[ \mathbb{P}(B) = \dfrac{13}{52} = \dfrac{1}{4}. \]
- \(\mathbb{P}(A \cap B)\) is the probability the card is both a Queen and a heart. Since there is exactly one Queen of hearts, it follows that there is only one card that satisfies the conditions:
\[ \mathbb{P}(A \cap B) = \dfrac{1}{52}. \]
Now you can check whether the multiplication rule holds. Here,
\[ \mathbb{P}(A)\mathbb{P}(B) = \left(\dfrac{1}{13}\right)\left(\dfrac{1}{4}\right) = \dfrac{1}{52} \]
Since \[ \mathbb{P}(A \cap B) = \dfrac{1}{52} = \mathbb{P}(A)\mathbb{P}(B), \]
the events \(A\) and \(B\) are independent.
Disjoint events
When working with probability, it’s important to understand the difference between disjoint events and independent events. These terms are often confused, but they describe fundamentally different relationships between events.
- Disjoint events (also called mutually exclusive events) are events that cannot occur at the same time. If \(A\) and \(B\) are disjoint, then:
\[ \mathbb{P}(A \cap B) = 0. \] For example, flipping a coin once cannot result in both heads and tails happening at the same time.
- Independent events are events where the outcome of one does not affect the probability of the other. If \(A\) and \(B\) are independent, then:
\[ \mathbb{P}(A \cap B) = \mathbb{P}(A)\mathbb{P}(B). \]
For example, flipping a coin and rolling a die are independent events because the result of the coin flip does not change the probability of the die roll.
| Feature | Disjoint Events | Independent Events |
|---|---|---|
| Can both events occur at the same time? | No | Yes |
| Formula | \(\mathbb{P}(A \cap B) = 0\) | \(\mathbb{P}(A \cap B) = \mathbb{P}(A) \mathbb{P}(B)\) |
| Relationship between events | If one happens, the other cannot | One happening does not change the probability of the other |
Table 1: Table comparing the key differences between disjoint and independent events.
Quick check problems
In the following questions, enter your answers as simplified fractions.
- Are the following statements true or false?
Conditional probability is only defined when the probability of the given event is greater than zero.
If \(\mathbb{P}(A \mid B) = \mathbb{P}(A)\), then \(A\) and \(B\) are independent.
Disjoint events are always independent.
- A card is drawn at random from a standard deck of 52 cards.
Given that the card is red, what is the probability that it is a heart?
Given that the card is a face card, what is the probability that it is a King?
A bag contains 3 green sweets and 2 yellow sweets. Two are picked without replacement. What is the probability that both are green?
You are given that \(\mathbb{P}(A) = 0.4\), \(\mathbb{P}(B) = 0.5\), and \(\mathbb{P}(A \cap B) = 0.2\). Are events \(A\) and \(B\) independent or dependent?
For each pair of events below, decide whether they are disjoint, independent, both, or neither.
\(A\) = “rolling an even number”, \(B\) = “rolling a number greater than 4” (on a fair six-sided die)
\(A\) = “drawing a Queen”, \(B\) = “drawing a face card” (from a standard deck)
Further reading
For more questions on the subject, please go to Questions: Conditional probability.
For more applications of conditional probability, including the central result of the subject, please go to Guide: Law of total probability and Bayes’ theorem.
Version history and licensing
v1.0: initial version created 05/25 by Sophie Chowgule as part of a University of St Andrews VIP project.