Lecture Notes: Probability

Preliminaries

Name tags
Sign in
Announce review day, lab, written assignment, and course evals

Course Content

Review 1

Question: How many ways are there to roll a total of 7 with two standard six-sided dice?
Answer: Six different ways. Illustrate with a 6x6 matrix.

Review 2

Review day for final exam. Review written assignments, quizzes, etc.

Motivation

Last week, we learned about counting. We can use counting to determine the probability of different outcomes.
Write: Definition: An event’s probability is the frequency with which an outcome is expected.
For example: What is the probability of rolling a total of 7 with two six-sided dice?
We counted there were six different ways to roll 7.
Overall, there are \( 6 \cdot 6 = 36 \) possible rolls.
So the probability of rolling a total of 7 is \( \frac{6}{36} = \frac{1}{6} \approx 0.167 \approx 16.7 \% \)
Percentages just represent ratios in terms of expected events out of 100. This makes it easier to compare probabilities.
16.7% probability means that if we rerolled the dice 100 times, we would expect to roll a total of 7 in approximately 16.7 cases.
Of course, due to randomness, you may not get exactly what you expect.
As we will see, it is often straightforward to calculate probabilities for chance-based games using counting.
However, we can also consider the probabilities of complex, real-world events.
For example, what is the probability that it will rain tomorrow?
Check phone: My phone says there is a X% chance of rain.
Weather predictions come from computer simulations run on data from satellite imagery and other sources.

Definitions

Write: Definition: A trial is a repeatable procedure, which results in one of a well-defined set of outcomes.
For example, rolling a pair of dice, drawing a marble from a jar, or flipping a coin.
Let’s think about the outcomes for each of these trials:
- Rolling a pair of dice. Outcome: total.
- Drawing a marble from a jar. Outcome: color of the marble.
- Flipping a coin. Outcome: heads or tails.
We will focus on finite sets of possible outcomes.
Write: Definition: An experiment is a sequence of trials.
You can repeat trials and you can repeat experiments. So it’s up to you what you consider a trial or an experiment.
For example, imagine flipping a coin four times.
- Is that a single trial, with 16 possible outcomes?
- Or is that one experiment, with four trials? Each trial has 2 possible outcomes.
Write: Definition: The sample space of a trial is the set of all possible outcomes.
For example:
- The sample space of rolling a die: S = {1, 2, 3, 4, 5, 6}
- The sample space of rolling a pair of dice: S = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
- The sample space of a coin flip: S = {Heads, Tails}
Write: Definition: An event is any set of outcomes: a subset of the sample space.
Examples of events:
- Rolling an even face of a die: A = {2, 4, 6}
- Rolling an even total using a pair of dice: A = {2, 4, 6, 8, 10, 12}
- Getting heads when flipping a coin: A = {Heads}
Write: Definition: Consider a trial with a finite sample space S. A probability function is a function \( \text{Pr: } \mathcal{P}(S) \rightarrow \mathbb{R} \) that assigns probabilities to events.
Note that the set of possible events of a trial is the power set of the sample space.
Write: Probability functions must satisfy the axioms of finite probability:
- Axiom 1: For any event A, \( 0 \leq \text{Pr}(A) \leq 1 \).
  - The probability of all events is between 0 and 1 inclusive.
- Axiom 2: \( \text{Pr}(\emptyset) = 0 \) and \( \text{Pr}(S) = 1\).
  - In any trial, one of the outcomes in S must occur.
  - For finite sample spaces, probability 0 means “impossible” and 1 means “certain.”
  - Ask: What is the probability of {Heads, Tails}?
- Axiom 3: If \( A, B \subseteq S \) and \( A \cap B = \emptyset \), then \( \text{Pr}(A \cup B) = \text{Pr}(A) + \text{Pr}(B) \).
  - The probability of the union of two disjoint events is the sum of their probabilities.
  - Ask: Using a six-sided die, what is the probability of rolling a 2 or 3?
  - Ask: Are these events disjoint? \( A = \{1, 2, 3\}, B = \{3, 4 \} \)

Complement

Write: Definition: \( \bar{A} \) is the complement or negation of event A: the set of possible outcomes when A doesn’t happen.
For example:
- The sample space of rolling a die: S = {1, 2, 3, 4, 5, 6}
- The event of rolling an even face: A = {2, 4, 6}
- The complement of this event: \( \bar{A} \) = {1, 3, 5}
Write: Theorem: The probability of event A not happening is \( \text{Pr}(\bar{A}) = 1 - \text{Pr}(A) \).
Proof: S is the set of all possible outcomes. If the event A does not happen, then the event \( S - A = \bar{A} \) does happen. The events A and \( \bar{A} \) are disjoint.

\[\begin{aligned} 1 &= \text{Pr}(S) & \text{ (by Axiom 2)} \\ &= \text{Pr}(A \cup \bar{A}) \\ &= \text{Pr}(A) + \text{Pr}(\bar{A}) & \text{ (by Axiom 3)} \\ 1 - \text{Pr}(A) &= \text{Pr}(\bar{A}) & \text{ (rearranging)} \\ \end{aligned}\]

Let’s apply this theorem to an example.
Ask: If you roll a pair of dice, what is the probability of not getting snake eyes (two 1s)?
- Answer: \( \text{Pr(SnakeEyes)} = \frac{1}{36} \). So the probability of not getting snake eyes is \( 1 - \frac{1}{36} = \frac{35}{36} \).

Theorems (Skipping Proofs)

Skip the proofs, but give examples to illustrate.
Write: Theorem: If one event is a subset of another, the first has no greater probability than the second. That is, if \( A, B \subseteq S \) and \( A \subseteq B \), then \( \text{Pr}(A) \leq \text{Pr}(B) \).
Proof: Consider the disjoint sets \( A \) and \( B - A \). Since \( A \subseteq B \):

\[\begin{aligned} \text{Pr}(B) &= \text{Pr}(A \cup (B - A)) \\ &= \text{Pr}(A) + \text{Pr}(B - A) & \text{ (by Axiom 3)} \\ \text{Pr}(A) &= \text{Pr}(B) - \text{Pr}(B - A) & \text{ (rearranging)} \end{aligned}\]

By Axiom 1, \( \text{Pr}(B - A) \geq 0 \). Thus, \( \text{Pr}(A) \leq \text{Pr}(B) \).
For example, consider events:
- P: there is precipitation tomorrow
- R: there is rain tomorrow
- \( R \subseteq P \), since rain is just one type of precipitation. So the probability of rain must be less than or equal to the probability of precipitation.
Write: Theorem: The probability that one of a set of disjoint events occurs is the sum of their individual probabilities. That is, suppose \( A_1, A_2, \ldots, A_n \subseteq S \) are all disjoint. Then: \[ \text{Pr}\left( \bigcup_{i=1}^n A_i \right) = \sum_{i=1}^n \text{Pr}(A_i) \]
Proof: Skipping for time, but it is a nice proof by induction (page 299).
For example, consider rolling a six-sided die, and the disjoint events: \[ A_1 = \{ 1 \}, A_2 = \{ 2, 3 \}, A_3 = \{ 5, 6 \} \]

\[\begin{aligned} \text{Pr}(A_1 \cup A_2 \cup A_3) &= \text{Pr}(A_1) + \text{Pr}(A_2) + \text{Pr}(A_3) \\ &= \frac{1}{6} + \frac{2}{6} + \frac{2}{6} \\ &= \frac{5}{6} \end{aligned}\]

Write: Theorem: Inclusion-Exclusion Principle. If A and B are any two events (not necessarily disjoint), the probability that one or the other will happen is the sum of their probabilities minus the probability of their intersection: \[ \text{Pr}(A \cup B) = \text{Pr}(A) + \text{Pr}(B) - \text{Pr}(A \cap B) \]
For example, consider rolling a six-sided die, and the non-disjoint events: \[ A = \{ 1, 2, 3 \}, B = \{ 3, 4 \} \]

\[\begin{aligned} \text{Pr}(A \cup B) &= \text{Pr}(A) + \text{Pr}(B) - \text{Pr}(A \cap B) \\ &= \frac{3}{6} + \frac{2}{6} - \frac{1}{6} \\ &= \frac{4}{6} \end{aligned}\]

Equally Likely Outcomes

Many phenomena have equally likely outcomes. For example, flipping a coin or rolling a dice.
Write: For a trial with n equally likely outcomes, we can assign equal probability \( \frac{1}{n} \) to each outcome.
Write: Theorem: Probability Function for Equally Likely Outcomes. Consider an experiment with finite sample space S, where each outcome in S is equally likely. Define, for any event \( E \subseteq S \), the probability function: \[ \text{Pr}(E) = \frac{\vert E \vert}{\vert S \vert} \]
Before we prove that this is indeed a probability function, let’s see a couple examples. We’ll consider the case of rolling a six-sided die.
- If \( E = \{1\} \), \( \text{Pr}(E) = \frac{1}{6} \)
- If \( E = \{1, 2\} \), \( \text{Pr}(E) = \frac{2}{6} \)
- If \( E = \{1, 2, 3, 4, 5, 6\} \), \( \text{Pr}(E) = \frac{6}{6} \)
Let’s prove that Pr is a probability function.
Proof: We will check the axioms of finite probability.
1. We will prove that for any event A, that \( 0 \leq \text{Pr}(A) \leq 1 \), where \( \text{Pr}(A) = \frac{\vert A \vert}{\vert S \vert} \). Both numerator and denominator of \( \frac{\vert A \vert}{\vert S \vert} \) are nonnegative, so \( 0 \leq \frac{\vert A \vert}{\vert S \vert} \). And since \( A \subseteq S \), \( \frac{\vert A \vert}{\vert S \vert} \leq 1 \).
2. \( \text{Pr}(\emptyset) = \frac{\vert \emptyset \vert}{\vert S \vert} = 0 \) and \( \text{Pr}(S) = \frac{\vert S \vert}{\vert S \vert} = 1\).
3. If A and B are disjoint:

\[\begin{aligned} \text{Pr}(A \cup B) &= \frac{\vert A \cup B \vert}{\vert S \vert} \\ &= \frac{\vert A \vert + \vert B \vert}{\vert S \vert} & \text{ (by the Sum Rule)} \\ &= \frac{\vert A \vert}{\vert S \vert} + \frac{\vert B \vert}{\vert S \vert} \\ &= \text{Pr}(A) + \text{Pr}(B) \end{aligned}\]

Birthday Problem

Write: In a randomly selected set of n ≤ 365 people, what is the probability that at least two have the same birthday?
We assume that all 365 birthdays in a nonleap year are equally likely, that no one is born on February 29, and that the births are independent of each other—no twins.
Answer: It is easier to calculate the probability of all birthdays being unique. Taking the complement of this will give our answer.
Consider n = 2. The 1st person can be born any day, but the 2nd person must be born on one of the 364 other days. So the probability of unique birthdays is \( \frac{364}{365} \).
Consider n = 3. The probability that a third person’s birthday is different than the first two’s is \( \frac{363}{365} \). So the probability that all three are different is \( \frac{364}{365} \cdot \frac{364}{365} \).
In general, after i people have been added, if all their birthdays are unique, then 365 - i birthdays remain unclaimed. So the probability the next person’s birthday will be different is \( \frac{365 - i}{365} \). So the probability that all n birthdays are different is:

\[\begin{aligned} \prod_{i=0}^{n-1} \frac{365-i}{365} &= \frac{365 \cdot 364 \cdot \ldots \cdot (365 - (n - 1))}{365^n} \\ &= \frac{365!}{(365 - n)! \cdot 365^n} & \text{ (falling factorial)} \end{aligned}\]

The probability that two people do share a birthday is 1 minus this number.
Let’s see the probability of at least two people having the same birthday for different values of n:
- n = 10, ≈12%
- n = 23, ≈51%
- n = 30, ≈71%
- n = 40, ≈89%
- n = 50, ≈97%

Independent Events

Ask: Let n be a large number. If a coin is flipped n times and comes up heads the first n − 1 times, what is the probability that the nth flip comes up heads?
Answer: The probability is still \( \frac{1}{2} \). A given coin flip doesn’t depend on previous coin flips.
Write: Definition: When the knowledge of whether an event A occurred gives no information about whether another event B occurred, A and B are said to be independent. Mathematically, events A and B are independent if and only if \[ \text{Pr}(A \cap B) = \text{Pr}(A) \cdot \text{Pr}(B) \]
Thinking that independent events (like coin flips) depend on each other is known as the Gambler’s Fallacy.
For example, consider rolling two six-sided dice:
- \( S = \{ \langle i, j \rangle : i, j \in \{1, 2, 3, 4, 5, 6\} \} \)
- Event A describes outcomes where the first die is 1. \(\text{Pr}(A) = \frac{1}{6} \) and \( A = \{ \langle 1, j \rangle : j \in \{1, 2, 3, 4, 5, 6\} \}\)
- Event B describes outcomes where the second die is 2. \(\text{Pr}(B) = \frac{1}{6} \) and \( A = \{ \langle i, 2 \rangle : i \in \{1, 2, 3, 4, 5, 6\} \}\)
- Event \(A \cap B = \{ \langle 1, 2 \rangle \} \) and \( \text{Pr}(A \cap B) = \frac{1}{36} \)
- We know that events A and B are independent because: \[ \text{Pr}(A) \cdot \text{Pr}(B) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36} \]
For an example of non-independent events, consider events:
- Event A describes outcomes where the first die is 3.
- Event B describes outcomes where the second die is greater than the first.

Introduction to Conditional Probability (If Time)

Ask: If you roll two dice, what is the probability of rolling two 1s (i.e., snake eyes)?
What if you roll the dice, and I tell you:
- One of the dice is odd? The probability is greater.
- One of the dice is even? The probability is zero.
Write: Definition: A conditional probability is the probability that an event will occur, on the condition that some other event occurs. Suppose A and B are events in sample space S, and \(\text{Pr}(B) > 0\). The conditional probability of A given B is: \[ \text{Pr}(A \vert B) = \frac{\text{Pr}(A \cap B)}{\text{Pr}(B)} \]
Let’s apply this to our snake eyes example.
- Event A describes rolling two 1s.
- Event B describes at least one of the dice being odd.
Draw: A table showing the 36 possible rolls, coloring events A and B.

\[\begin{aligned} \text{Pr}(A \cap B) &= \frac{1}{36} \\ \text{Pr}(B) &= \frac{27}{36} \\ \text{Pr}(A \vert B) &= \frac{\text{Pr}(A \cap B)}{\text{Pr}(B)} = \frac{1}{27} \\ \end{aligned}\]

Write: Theorem: For events A and B that both have nonzero probability, A and B are independent if and only if \( \text{Pr}(A \vert B) = \text{Pr}(A) \)
Intuitively, this means that knowing B doesn’t affect the probability of A.
Proof, Part 1: Suppose A and B are independent events with nonzero probability. By the definition of conditional probability:

\[\begin{aligned} \text{Pr}(A \vert B) &= \frac{\text{Pr}(A \cap B)}{\text{Pr}(B)} \\ &= \frac{\text{Pr}(A) \cdot \text{Pr}(B)}{\text{Pr}(B)} & \text{ (by the definition of independence)} \\ &= \text{Pr}(A) \\ \end{aligned}\]

Proof, Part 2:

\[\begin{aligned} \text{Pr}(A \vert B) &= \text{Pr}(A) \\ \frac{\text{Pr}(A \cap B)}{\text{Pr}(B)} &= \text{Pr}(A) & \text{ (by the definition of conditional probability)} \\ \text{Pr}(A \cap B) &= \text{Pr}(A) \cdot \text{Pr}(B) & \text{ (the definition of independence)} \\ \end{aligned}\]

Ask: Identify the flaw in the prosecutor’s argument:
- A driver is accused of speeding. The prosecutor notes that the driver had a radar detector in their car. The prosecutor cites data that 80% of speeders have radar detectors, and claims the driver is 80% likely to be guilty of speeding.
Answer: The data represents the probability Pr(detector|speeding) = 0.8
However, the relevant probability is unknown: Pr(speeding|detector) = ?
Perhaps 80% of all drivers own radar detectors.
See Chapter 27 for other interesting examples of conditional probability.