Lecture Notes: Probability

Preliminaries

• Name tags
• Sign in
• Announce review day, lab, written assignment, and course evals

Course Content

Tuesday Review

• Question: How many ways are there to roll a total of 7 with two standard six-sided dice?
• Answer: Six different ways. Illustrate with a 6x6 matrix.

Thursday Review

Review day for final exam. Review written assignments, quizzes, etc.

Motivation

• Last week, we learned about counting. We can use counting to determine the probability of different outcomes.
• Write: Definition: An event’s probability is the frequency with which an outcome is expected.
• For example: What is the probability of rolling a total of 7 with two six-sided dice?
• We counted there were six different ways to roll 7.
• Overall, there are $6 \cdot 6 = 36$ possible rolls.
• So the probability of rolling a total of 7 is $\frac{6}{36} = \frac{1}{6} \approx 0.167 \approx 16.7 \%$
• Percentages just represent ratios in terms of expected events out of 100. This makes it easier to compare probabilities.
• 16.7% probability means that if we rerolled the dice 100 times, we would expect to roll a total of 7 in approximately 16.7% of cases.
• As we will see, it is often straightforward to calculate probabilities for chance-based games using counting.
• However, we can also consider the probabilities of complex, real-world events.
• For example, what is the probability that it will rain tomorrow?
• Check phone: My phone says there is a X% chance of rain.
• Weather predictions come from computer simulations run on data from satellite imagery and other sources.

Definitions

• Write: Definition: A trial is a repeatable procedure, which results in one of a well-defined set of outcomes.
• For example, rolling a pair of dice, drawing a marble from a jar, or flipping a coin.
• Let’s think about the outcomes for each of these trials:
  • Rolling a pair of dice. Outcome: total.
  • Drawing a marble from a jar. Outcome: color of the marble.
  • Flipping a coin. Outcome: heads or tails.
• We will focus on finite sets of possible outcomes.
• Write: Definition: An experiment is a sequence of trials.
• You can repeat trials and you can repeat experiments. So it’s up to you what you consider a trial or an experiment.
• For example, imagine flipping a coin four times.
  • Is that a single trial, with 16 possible outcomes?
  • Or is that one experiment, with four trials? Each trial has 2 possible outcomes.
• Write: Definition: The sample space of a trial is the set of all possible outcomes.
• For example:
  • The sample space of rolling a die: S = {1, 2, 3, 4, 5, 6}
  • The sample space of rolling a pair of dice: S = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
  • The sample space of a coin flip: S = {Heads, Tails}
• Write: Definition: An event is any set of outcomes: a subset of the sample space. The set of possible events of a trial is the power set of the sample space.
• Examples of events:
  • Rolling an even face of a die: A = {2, 4, 6}
  • Rolling an even total using a pair of dice: A = {2, 4, 6, 8, 10, 12}
  • Getting heads when flipping a coin: A = {Heads}
• Write: Definition: Consider a trial with a finite sample space S. A probability function is a function $\text{Pr: } \mathcal{P}(S) \rightarrow \mathbb{R}$ that assigns probabilities to events.
• Write: Probability functions must satisfy the axioms of finite probability:
  • Axiom 1: For any event A, $0 \leq \text{Pr}(A) \leq 1$.
    • The probability of all events is between 0 and 1 inclusive.
  • Axiom 2: $\text{Pr}(\emptyset) = 0$ and $\text{Pr}(S) = 1$.
    • In any trial, one of the outcomes in S must occur.
    • For finite sample spaces, probability 0 means “impossible” and 1 means “certain.”
    • Ask: What is the probability of {Heads, Tails}?
  • Axiom 3: If $A, B \subseteq S$ and $A \cap B = \emptyset$, then $\text{Pr}(A \cup B) = \text{Pr}(A) + \text{Pr}(B)$.
    • The probability of the union of two disjoint events is the sum of their probabilities.
    • Ask: Using a six-sided die, what is the probability of rolling a 2 or 3?
    • Ask: Are these events disjoint? $A = \{1, 2, 3\}, B = \{3, 4 \}$

Complement

• Write: Definition: $\bar{A}$ is the complement or negation of event A: the set of possible outcomes when A doesn’t happen.
• For example:
  • The sample space of rolling a die: S = {1, 2, 3, 4, 5, 6}
  • The event of rolling an even face: A = {2, 4, 6}
  • The complement of this event: $\bar{A}$ = {1, 3, 5}
• Write: Theorem: The probability of event A not happening is $\text{Pr}(\bar{A}) = 1 - \text{Pr}(A)$.
• Proof: S is the set of all possible outcomes. If the event A does not happen, then the event $S - A = \bar{A}$ does happen. The events A and $\bar{A}$ are disjoint.

• Let’s apply this theorem to an example.
• Ask: If you roll a pair of dice, what is the probability of not getting snake eyes (two 1s)?
  • Answer: $\text{Pr(SnakeEyes)} = \frac{1}{36}$. So the probability of not getting snake eyes is $1 - \frac{1}{36} = \frac{35}{36}$.

Theorems (Skipping Proofs)

• Skip the proofs, but give examples to illustrate.
• Write: Theorem: If one event is a subset of another, the first has no greater probability than the second. That is, if $A, B \subseteq S$ and $A \subseteq B$, then $\text{Pr}(A) \leq \text{Pr}(B)$.
• Proof: Consider the disjoint sets $A$ and $B - A$. Since $A \subseteq B$:

• By Axiom 1, $\text{Pr}(B - A) \geq 0$. Thus, $\text{Pr}(A) \leq \text{Pr}(B)$.
• For example, consider events:
  • P: there is precipitation tomorrow
  • R: there is rain tomorrow
  • $R \subseteq P$, since rain is just one type of precipitation. So the probability is rain must be less than the probability of precipitation.
• Write: Theorem: The probability that one of a set of disjoint events occurs is the sum of their individual probabilities. That is, suppose $A_1, A_2, \ldots, A_n \subseteq S$ are all disjoint. Then: $$\text{Pr}\left( \bigcup_{i=1}^n A_i \right) = \sum_{i=1}^n \text{Pr}(A_i)$$
• Proof: Skipping for time, but it is a nice proof by induction (page 299).
• For example, consider rolling a six-sided die, and the disjoint events: $$A_1 = \{ 1 \}, A_2 = \{ 2, 3 \}, A_3 = \{ 5, 6 \}$$

• Write: Theorem: Inclusion-Exclusion Principle. If A and B are any two events (not necessarily disjoint), the probability that one or the other will happen is the sum of their probabilities minus the probability of their intersection: $$\text{Pr}(A \cup B) = \text{Pr}(A) + \text{Pr}(B) - \text{Pr}(A \cap B)$$

• For example, consider rolling a six-sided die, and the non-disjoint events: $$A = \{ 1, 2, 3 \}, B = \{ 3, 4 \}$$

Equally Likely Outcomes

• Many phenomena have equally likely outcomes. For example, flipping a coin or rolling a dice.
• Write: For a trial with n equally likely outcomes, we can assign equal probability $\frac{1}{n}$ to each outcome.
• Write: Theorem: Probability Function for Equally Likely Outcomes. Consider an experiment with finite sample space S, where each outcome in S is equally likely. Define, for any event $E \subseteq S$, the probability function: $$\text{Pr}(E) = \frac{\vert E \vert}{\vert S \vert}$$
• Before we prove that this is indeed a probability function, let’s see a couple examples. We’ll consider the case of rolling a six-sided die.
  • If $E = \{1\}$, $\text{Pr}(E) = \frac{1}{6}$
  • If $E = \{1, 2\}$, $\text{Pr}(E) = \frac{2}{6}$
  • If $E = \{1, 2, 3, 4, 5, 6\}$, $\text{Pr}(E) = \frac{6}{6}$
• Let’s prove that Pr is a probability function.
• Proof: We will check the axioms of finite probability.
  1. We will prove that for any event A, that $0 \leq \text{Pr}(A) \leq 1$, where $\text{Pr}(A) = \frac{\vert A \vert}{\vert S \vert}$. Both numerator and denominator of $\frac{\vert A \vert}{\vert S \vert}$ are nonnegative, so $0 \leq \frac{\vert A \vert}{\vert S \vert}$. And since $A \subseteq S$, $\frac{\vert A \vert}{\vert S \vert} \leq 1$.
  2. $\text{Pr}(\emptyset) = \frac{\vert \emptyset \vert}{\vert S \vert} = 0$ and $\text{Pr}(S) = \frac{\vert S \vert}{\vert S \vert} = 1$.
  3. If A and B are disjoint:

Birthday Problem

• Write: In a randomly selected set of n ≤ 365 people, what is the probability that at least two have the same birthday?
• We assume that all 365 birthdays in a nonleap year are equally likely, that no one is born on February 29, and that the births are independent of each other—no twins.
• Answer: It is easier to calculate the probability of all birthdays being unique. Taking the complement of this will give our answer.
• Consider n = 2. The 1st person can be born any day, but the 2nd person must be born on one of the 364 other days. So the probability of unique birthdays is $\frac{364}{365}$.
• Consider n = 3. The probability that a third person’s birthday is different than the first two’s is $\frac{363}{365}$. So the probability that all three are different is $\frac{364}{365} \cdot \frac{364}{365}$.
• In general, after i people have been added, if all their birthdays are unique, then 365 - i birthdays remain unclaimed. So the probability the next person’s birthday will be different is $\frac{365 - i}{365}$. So the probability that all n birthdays are different is:

• The probability that two people do share a birthday is 1 minus this number.
• Let’s see the probability of at least two people having the same birthday for different values of n:
  • n = 10, ≈12%
  • n = 23, ≈51%
  • n = 30, ≈71%
  • n = 40, ≈89%
  • n = 50, ≈97%

Independent Events

• Ask: Let n be a large number. If a coin is flipped n times and comes up heads the first n − 1 times, what is the probability that the nth flip comes up heads?
• Answer: The probability is still $\frac{1}{2}$. A given coin flip doesn’t depend on previous coin flips.
• Write: Definition: When the knowledge of whether an event A occurred gives no information about whether another event B occurred, A and B are said to be independent. Mathematically, events A and B are independent if and only if $$\text{Pr}(A \cap B) = \text{Pr}(A) \cdot \text{Pr}(B)$$
• Thinking that independent events (like coin flips) depend on each other is known as the Gambler’s Fallacy.
• For example, consider rolling two six-sided dice:
  • $S = \{ \langle i, j \rangle : i, j \in \{1, 2, 3, 4, 5, 6\} \}$
  • Event A describes outcomes where the first die is 1. $\text{Pr}(A) = \frac{1}{6}$ and $A = \{ \langle 1, j \rangle : j \in \{1, 2, 3, 4, 5, 6\} \}$
  • Event B describes outcomes where the second die is 2. $\text{Pr}(B) = \frac{1}{6}$ and $A = \{ \langle i, 2 \rangle : i \in \{1, 2, 3, 4, 5, 6\} \}$
  • Event $A \cap B = \{ \langle 1, 2 \rangle \}$ and $\text{Pr}(A \cap B) = \frac{1}{36}$
  • We know that events A and B are independent because: $$\text{Pr}(A) \cdot \text{Pr}(B) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36}$$
• For an example of non-independent events, consider events:
  • Event A describes outcomes where the first die is 3.
  • Event B describes outcomes where the second die is greater than the first.