CHAPTER
6

Introduction to Probability

In This Chapter

• Distinguish between classical, empirical, and subjective probability
• Use frequency distributions to calculate probability
• Understand the relationship between events
• Demonstrate the intersection and union of events using a Venn diagram

As we leave the happy world of descriptive statistics, you may feel like you’re ready to take on the challenge of inferential statistics. But before we enter that realm, we need to arm ourselves with probability theory. Accurately predicting the probability that an event will occur has widespread applications. For instance, the gaming industry uses probability theory to set odds for lotteries, card games, and sporting events.

The focus of this chapter is to start with the basics of probability, after which we will gently proceed to more complex concepts in Chapters 7 and 8. We’ll discuss different types of probabilities and how to calculate the probability of simple events. We’ll rely on data from frequency distributions to examine the likelihood of an event. So pull up a chair and let’s roll those dice!

What is Probability?

Probability is a measure of likelihood that an event will happen in the future. It takes a value between 0 and 1, often expressed as a percentage. The closer the probability is to 1, the more likely the event will happen in the future, and the closer the probability is to 0, the less likely the event will happen. Probability cannot be negative and cannot be greater than 1. The weather forecast is an example of probability, like when you hear in the news that there is a 60 percent chance of rain tomorrow. The higher the number is, the more likely it is to rain tomorrow, while the lower the number is, the less likely it is to rain tomorrow.

If you are absolutely sure that an event will occur, then the probability of this event occurring is 1. For example, in rolling a single die, what is the probability of getting a number less than 7? It is 1 because I’m 100 percent certain that the number I’ll get is less than 7. It could be 1, 2, 3, 4, 5, or 6, and all of these are less than 7. On the contrary, the probability of an impossible event is zero. For example, in rolling the single die, what is the probability of getting a 7? There is no chance the number I roll can be 7, so it is an impossible event and its probability of occurring is 0.

Before we go any further, we need to tackle some new statistics jargon. The following terms are widely used when talking about probability:

• Experiment The process of measuring or observing an activity for the purpose of collecting data. An example is rolling a single die.
• Outcome A particular result of an experiment. An example is getting a head when flipping a coin.
• Sample space All the possible outcomes of the experiment. In rolling a single die, the sample space is {1, 2, 3, 4, 5, and 6}. Statistics people like to put { } around the sample space values because they think it looks cool.
• Event One or more outcomes of an experiment, which are a subset of the sample space. For example, in rolling a single die, getting an even number is an event.

Now that we know what probability is, let’s look at the three main methods of measuring it: classical, empirical, and subjective.

Classical Probability

Classical probability refers to a situation when we know the number of possible outcomes of the event of interest and the total number of possible outcomes. It is based on the assumption that each outcome of the experiment is equally likely to occur. The probability of an event can be calculated using the following equation:

P(A) =

where:

P(A): the probability that Event A will occur.

For example, what is the probability of randomly selecting a Jack from a deck of cards? P(Jack)= . That’s because we have 4 jacks in the deck and 52 cards all together. Now, let me test your solitaire skills. What is the probability of randomly selecting a face from a deck of cards? I hear you saying P(Face) = . You are correct! I guess you play solitaire a lot!

DEFINITION

Classical probability requires that you know the number of outcomes that pertain to a particular event of interest. You also need to know the total number of possible outcomes in the sample space.

To use classical probability, you need to understand the underlying process so you can determine the number of outcomes associated with the event. You also need to be able to count the total number of possible outcomes in the sample space. As you will see next, this may not always be possible.

Empirical Probability

When we don’t know enough about the underlying process to determine the number of outcomes associated with an event, we rely on empirical probability. This type of probability observes the number of occurrences of an event through an experiment and calculates the probability from a relative frequency distribution. Therefore:

P(A) =

DEFINITION

Empirical probability requires that you count the frequency that an event occurs through an experiment and calculate the probability from the relative frequency distribution.

One example of empirical probability is to answer the age-old question “What is the probability that John, Bob’s son, will get out of bed in the morning for school after his first wake-up call?” Because Bob cannot understand the underlying process of why a teenager will resist getting out of bed before 2 P.M., he needs to rely on empirical probability. The following table indicates the number of wake-up calls John required over 20 school days.

John’s Wake-Up Calls (Previous 20 School Days)

We can summarize this data with a relative frequency distribution.

Relative Frequency Distribution for John’s Wake-Up Calls

Based on these observations, if Event A = John getting out of bed on the first wake-up call, then P(A) = 0.15.

RANDOM THOUGHTS

The probability that you will win a typical state lottery, where you correctly choose 6 out of 49 numbers, is approximately 0.00000007, or 1 out of 14 million. This is calculated using classical probability. Compare this to the probability that you will be struck by lightning once during your lifetime (assume 80 years), which is 0.000083 or 1 out of 12,000 (source: www.lightningsafety.noaa.gov/odds.shtml).

If I choose to run another 20-day experiment of John’s waking behavior, I would most likely see different results than those in the previous table. However, if I were to observe 100 days of this data, the relative frequencies would approach the true or classical probabilities of the underlying process. This pattern is known as the law of large numbers.

To demonstrate the law of large numbers, let’s say I flip a coin three times and each time the result is heads. For this experiment, the empirical probability for the event heads is 100 percent. However, if I were to flip the coin 100 times, I would expect the empirical probability of this experiment to be much closer to the classical probability of 50 percent.

DEFINITION

The law of large numbers states that when an experiment is conducted a large number of times, the empirical probabilities of the process will converge to the classical probabilities.

Subjective Probability

We use subjective probability when classical and empirical probabilities are not available. Under these circumstances, we rely on experience and intuition to estimate the probabilities.

Examples where we would apply subjective probability are “What is the probability that Bob’s son Brian will ask to borrow Bob’s new car, which happens to have a 6-speed manual transmission, for his junior prom?” (97 percent), or “What is the probability that Bob’s new car will come back with all 6 gears in proper working order?” (18 percent). These probabilities are based on personal observations and experience. We need to use subjective probability in this situation because Bob’s car would never survive several of these “experiments.”

Relationship Between Events

To better understand probability, we need to know the meaning of three more statistics terms:

• Mutually Exclusive Events
• Independent Events
• Complementary Events

Mutually Exclusive Events

Events A and B are said to be mutually exclusive if they cannot occur at the same time during the experiment. For example, suppose my experiment is to roll a single die and my events are: Event A is to roll a 4 and Event B is to roll a 5. Can we get a 4 and a 5 simultaneously when we roll a single die once? No way, right? Since there is no way for these events to occur simultaneously, they are considered to be mutually exclusive. Another example is the result of your final exam as pass or fail. Can you pass and fail the same exam? No chance, so they are mutually exclusive.

DEFINITION

Two events are considered to be mutually exclusive if they cannot occur at the same time during the experiment.

Independent Events

Events A and B are said to be independent of each other if the occurrence of Event B has no effect on the probability of Event A. For example, suppose my experiment is to roll a single die twice and my events are: Event A is rolling a 2 and Event B is rolling a 6. The probability of getting a 6 when we roll the die for the second time is not affected by the probability of getting a 2 on the first roll. Therefore, these events are independent. Or for another example, in flipping a coin two separate times, the probability of getting a head on the second flip is not affected by the probability of getting a head on the first flip. Therefore, the events of getting a head the first time and a head the second time are independent events. On the other hand, dependent events depend on each other for their outcomes. For example, if your experiment is to draw marbles out of a bag and observe the color each time (without returning the marbles to the bag), then the probability of removing a certain color will change with each draw.

DEFINITION

Events A and B are said to be independent of each other if the occurrence of Event B has no effect on the probability of Event A.

Complementary Events

The complement to Event A is defined as all the outcomes in the sample space that are not part of Event A. It is denoted as A’ (pronounced A-prime), or not A. Since the probabilities of all events in the sample space add up to 1, then:

P(A) + P(A’) = 1. Equivalently, we can also say P(A’) = 1 – P(A).

This rule is very useful in calculating probability. Let’s apply it to some examples to see why. For instance, in rolling a single die, what is the probability of not getting a four? Not getting a 4 is the complement event of getting a 4, so P(4’) = 1 – P(4) = 1 – = . Or in another example, in drawing a card from a deck, what is the probability of not getting a Jack? P(Jack’) = 1 – P(Jack) = 1 – = . As you can see, using the complement rule makes the calculation easier and faster. Instead of having to count all the cards except Jacks, I can use the complement rule instead.

DEFINITION

The complement to Event A is defined as all the outcomes in the sample space that are not part of Event A. It is denoted as A’.

Union and Intersection of Events

Understanding the union and intersection of events is important when using probability rules, as we will see in the next chapter. So before we delve into probability rules, let’s see what these terms mean!

The Union of Events: A Marriage Made in Heaven

The union of Events A and B represents all the instances where either Event A or Event B or both occur and is denoted as A∪B. For example, I’m teaching two undergraduate courses this semester: Statistics and Finance. Let’s say that A represents students in my statistics class and B represents students in my finance class. A∪B includes all students who are in either my statistics class or in my finance class or in both. So any student in my statistics class but not in the finance class is part of the union. Likewise, any student in my finance class but not in the statistics class is part of the union. In addition, any student in both my statistics and my finance classes is part of the union.

DEFINITION

The union of Events A and B represents all the instances where either Event A or Event B or both occur.

The Intersection of Events

The intersection of Events A and B represents all the instances where both Event A and Event B occur at the same time and is denoted as A∩B. In our previous example, only students in both my statistics and finance classes are part of the intersection of the events. The probability of the intersection of two events is known as a joint probability.

DEFINITION

The intersection of Events A and B represents all the instances where both Event A and Event B occur at the same time.

To better illustrate the union and intersection of events, let’s look at the Venn diagram below.

Figure 6.1

Venn diagram.

A∪B includes all elements in either A or in B or in both, so any element in the whole diagram is part of the union. A∩B includes all elements in both A and B, so only those elements at the intersection of the two circles are part of the intersection.

TEST YOUR KNOWLEDGE

The Venn diagram was developed by English mathematician John Venn (1834–1923).

Practice Problems

1. Define each of the following as classical, empirical, or subjective probability.

a. The probability that the baseball player Derek Jeter will get a hit during his next bat.

b. The probability of drawing an Ace from a deck of cards.

c. The probability that I will shoot lower than a 90 during my next round of golf.

d. The probability of winning the next state lottery drawing.

e. The probability that I will finish writing this book before my deadline.

2. Identify whether each of the following are valid probabilities.

a. 65%

b. 1.9

c. 110%

d. -4.2

e. 0.75

f. 0

The Least You Need to Know

• Classical probability requires knowledge of the underlying process in order to count the number of possible outcomes of the event of interest.
• Empirical probability relies on historical data from a frequency distribution to calculate the likelihood that an event will occur.
• The law of large numbers states that when an experiment is conducted a large number of times, the empirical probabilities of the process will converge to the classical probabilities.
• The union of Events A and B represents the number of instances where either Event A or B or both occur.
• The intersection of Events A and B represents the number of instances where Events A and B occur at the same time.