Welcome to our lesson on "Probability Basics"! This topic is important for making predictions and understanding uncertainties in machine learning. By the end of this lesson, you'll know the essentials of probability and how to calculate probabilities for different events using Python.

Let's begin by understanding what probability is and why it matters.

Probability measures the likelihood of an event happening. Think of it as quantifying uncertainty. For example, what are the chances it will rain tomorrow, or that a flipped coin will land on heads?

Mathematically, the probability $P$ of an event $E$ happening can be represented as:

$P(E) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$

Probability ranges from 0 (an impossible event) to 1 (a certain event).

Let's use a deck of playing cards to understand probabilities practically. A standard deck has 52 cards. We'll calculate the probability of different events using these cards.

Imagine we want to calculate the probability of drawing an Ace from a standard deck.

Example: $P(\text{Ace}) = \frac{\text{Number of Aces}}{\text{Total cards}} = \frac{4}{52}$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of Aces in a standard deck
5aces = 4
6
7# Probability of drawing an Ace from a standard deck
8probability_of_ace = aces / total_cards
9print(f"Probability of drawing an Ace: {probability_of_ace:.2f}")  # Probability of drawing an Ace: 0.08

In the print function, .2f is a format specifier that limits the displayed output to 2 decimal places.

Next, let's calculate the probability of drawing a card from the Hearts suit.

Example: $P(\text{Hearts}) = \frac{\text{Number of Hearts}}{\text{Total cards}} = \frac{13}{52}$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of cards from the Hearts suit in a standard deck
5cards_of_suit = 13
6
7# Probability of drawing a card from the Hearts suit
8probability_of_suit = cards_of_suit / total_cards
9print(f"Probability of drawing a card from the Hearts suit: {probability_of_suit:.2f}")  # Probability of drawing a card from the Hearts suit: 0.25

This rule states that the probability of one of two mutually exclusive events happening is the sum of their individual probabilities.

Mutually exclusive events are events that cannot happen at the same time. In the context of drawing cards, drawing an Ace and a King are mutually exclusive because you can't draw both at the same time from a single draw.

For example, the probability of drawing either an Ace or a King:

$P(\text{Ace or King}) = P(\text{Ace}) + P(\text{King})$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of Aces in a standard deck
5aces = 4
6
7# Number of Kings in a standard deck
8kings = 4
9
10# Probability of drawing an Ace
11probability_of_ace = aces / total_cards
12
13# Probability of drawing a King
14probability_of_king = kings / total_cards
15
16# Probability of drawing an Ace or a King (mutually exclusive)
17probability_of_ace_or_king = probability_of_ace + probability_of_king
18print(f"Probability of drawing an Ace or a King (mutually exclusive): {probability_of_ace_or_king:.2f}")  # Probability of drawing an Ace or a King (mutually exclusive): 0.15

When two events are not mutually exclusive, they can happen at the same time. In this case, we need to subtract the probability of both events happening from the sum of their individual probabilities to avoid double-counting.

For example, consider the probability of drawing a card that is either an Ace or a Heart. Since the Ace of Hearts fits both categories, it needs to be subtracted once.

$P(\text{Ace or Heart}) = P(\text{Ace}) + P(\text{Heart}) - P(\text{Ace of Hearts})$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of Aces in a standard deck
5aces = 4
6
7# Number of Hearts in a standard deck
8hearts = 13
9
10# Number of Ace of Hearts in a standard deck
11ace_of_hearts = 1
12
13# Probability of drawing an Ace
14probability_of_ace = aces / total_cards
15
16# Probability of drawing a Heart
17probability_of_heart = hearts / total_cards
18
19# Probability of drawing an Ace of Hearts
20probability_of_ace_of_hearts = ace_of_hearts / total_cards
21
22# Probability of drawing an Ace or a Heart (non-mutually exclusive)
23probability_of_ace_or_heart = probability_of_ace + probability_of_heart - probability_of_ace_of_hearts
24print(f"Probability of drawing an Ace or a Heart (non-mutually exclusive): {probability_of_ace_or_heart:.2f}")  # Probability of drawing an Ace or a Heart (non-mutually exclusive): 0.31

The next rule applies to independent events, like drawing cards with replacement. Drawing with replacement means that after drawing a card, you place it back in the deck before drawing again. This ensures that the total number of cards remains the same for each draw.

The probability of both events happening is the product of their individual probabilities. For example, here is how we can calculate the probability of drawing a King and and Ace from two draws, given that we put the first card back and shuffle before the second draw.

Example: $P(\text{Ace and then King with replacement}) = P(\text{Ace}) \times P(\text{King})$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of Aces in a standard deck
5aces = 4
6
7# Number of Kings in a standard deck
8kings = 4
9
10# Probability of drawing an Ace
11probability_of_ace = aces / total_cards
12
13# Probability of drawing a King
14probability_of_king = kings / total_cards
15
16# Probability of drawing an Ace and then drawing a King with replacement
17probability_of_ace_and_king = probability_of_ace * probability_of_king
18print(f"Probability of drawing an Ace and then a King (independent, with replacement): {probability_of_ace_and_king:.4f}")  # Probability of drawing an Ace and then a King (independent, with replacement): 0.0059

When events are dependent, like drawing cards without replacement, the probability changes after the first event.

Example: If you draw an Ace first, there are now 51 cards left in the deck, and only 4 of them are Kings. So, the probability of drawing a King is no longer $\frac{4}{52}$, it is now $\frac{4}{51}$

$P(\text{Ace and then King without replacement}) = P(\text{Ace}) \times P(\text{King | Ace drawn})$

Here, $P(\text{King | Ace drawn})$ represents the conditional probability, i.e., the probability of drawing a King given that an Ace has already been drawn.

$P(\text{King | Ace drawn}) = \frac{\text{Number of Kings}}{\text{Total cards - 1}} = \frac{4}{51}$

Therefore,

$P(\text{Ace and then King without replacement}) = \frac{4}{52} \times \frac{4}{51}$

In Python:

Python
Copy to clipboard
1# Total number of cards in a standard deck
2total_cards = 52
3
4# Number of Aces in a standard deck
5aces = 4
6
7# Number of Kings in a standard deck
8kings = 4
9
10# Probability of drawing an Ace
11probability_of_ace = aces / total_cards
12
13# Probability of drawing a King given an Ace has been drawn
14probability_of_king_given_ace_drawn = kings / (total_cards - 1)
15
16# Probability of drawing an Ace and then drawing a King without replacement
17probability_of_ace_and_king_no_replace = probability_of_ace * probability_of_king_given_ace_drawn
18print(f"Probability of drawing an Ace and then a King (dependent, without replacement): {probability_of_ace_and_king_no_replace:.4f}")  # Probability of drawing an Ace and then a King (dependent, without replacement): 0.0060

Great job! Today we covered the basics of probability, including how to calculate probabilities for different events. We explored different types of events: mutually exclusive, non-mutually exclusive, independent, and dependent events to understand their probabilities.

By now, you should be comfortable with the basic probability concepts and calculations.

Next, you will move to a practice session where you will apply these concepts and solidify your understanding. Let's move on to practice!