This chapter explains the differences between probabilistic statistics and descriptive statistics, showing in which situations the probability theory must be used. The concepts and terminologies related to probabilistic statistics are presented here, as well as their practical application. By using the probability theory, it is possible to predict the occurrence of one or more events. This chapter also shows how combinatorial analysis can be used to calculate probabilities.
Probability theory; Events; Conditional probability; Bayes' theorem; Combinatorial analysis
Do you want to sell sugar water for the rest of your life, or do you want to come with me and change the world?
Steve Jobs
In the previous part of this book, we studied descriptive statistics, which describes and summarizes the main characteristics observed in a dataset through frequency distribution tables, charts, graphs, and summary measures, allowing the researcher to have a better understanding of the data.
Probabilistic statistics, on the other hand, uses the probability theory to explain how often certain uncertain events happen, in order to estimate or predict the occurrence of future events. For example, when rolling dice, we do not know for sure which value will appear, so, probability can be used to indicate the occurrence probability of a certain event.
According to Bruni (2011), the history of probability presumably started with the cave men. They needed to understand nature's uncertain phenomena better. In the 17th century, probability theory appeared to explain uncertain events. The study of probability evolved to help plan moves or develop strategies meant for gambling. Currently, it is also applied to the study of statistical inference, in order to generalize the data population.
This chapter has as its main objective to present the concepts and terminologies related to the probability theory, as well as their practical application.
An experiment consists in any observation or measure process. A random experiment is one that generates unpredictable results, so, if the process is repeated several times, it becomes impossible to predict the result. Flipping a coin and/or rolling dice are examples of random experiments.
Sample space S consists of all the possible results of an experiment.
For example, when flipping a coin, we can get head (H) or tail (T). Therefore, S = {H, T}. On the other hand, when rolling a die, the sample space is represented by S = {1, 2, 3, 4, 5, 6}.
An event is any subset of a sample space.
For example, event A only contains the even occurrences of rolling a die. Therefore, A = {2, 4, 6}.
Two or more events can form unions, intersections, and complements.
The union of two events A and B, represented by A ∪ B, results in a new event containing all the elements of A, B, or both, and can be illustrated according to Fig. 5.1.
The intersection of two events A and B, represented by A ∩ B, results in a new event containing all the elements that are simultaneously in A and B, and can be illustrated according to Fig. 5.2.
The complement of an event A, represented by Ac, is the event that contains all the points of S that are not in A, as shown in Fig. 5.3.
Two events A and B are independent when the probability of B happening is not conditional on event A happening. The concept of conditional probability will be discussed in Section 5.5.
Mutually excluding or exclusive events are those that do not have any elements in common, so, they cannot happen simultaneously. Fig. 5.4 illustrates two events A and B that are mutually exclusive.
The probability of a certain event A happening in sample space S is given by the ratio between the number of cases favorable to the event (nA) and the total number of possible cases (n):
The probability of an event A happening is a number between 0 and 1:
Sample space S has probability equal to 1:
The probability of an empty set (ϕ) occurring is null:
The probability of event A, event B or both happening can be calculated as follows:
If events A and B are mutually exclusive, that is, A ∩ B ≠ ϕ, the probability of one of them happening is equal to the sum of the individual probabilities:
Expression (5.6) can be extended to n events (A1, A2, …, An) that are mutually exclusive:
If Ac is A's complementary event, then:
If A and B are two independent events, the probability of them happening together is equal to the product of their individual probabilities:
Expression (5.9) can be extended to n independent events (A1, A2, …, An):
When events are not independent, we must use the concept of conditional probability. Considering two events A and B, the probability of A happening, given that B has already happened, is called conditional probability of A given B, and is represented by P(A | B):
An event A is considered independent of B if:
From the definition of conditional probability, the multiplication rule allows researcher to calculate the probability of the simultaneous occurrence of two events A and B as the probability of one of them multiplied by the conditional probability of the other, given that the first event has occurred:
The multiplication rule can be extended to three events A, B, and C:
This is only one of the six ways in which Expression (5.14) can be written.
Imagine that the probability of a certain event was calculated. However, new information was added to the process, so, the probability must be recalculated. The probability calculated initially is called a priori probability; the probability with the recently added information is called a posteriori probability. The calculation of the a posteriori probability is based on Bayes' Theorem and is described here.
Consider B1, B2, …, Bn mutually exclusive events, and P(B1) + P(B2) + … + P(Bn) = 1. A, on the other hand, is any given event that will happen jointly or as a consequence of one of the Bi events (i = 1, 2, …, n). The probability of a Bi event happening, given that A event has already happened, is calculated as follows:
where:
Combinatorial analysis is a set of procedures that calculates the number of different groups that can be formed by selecting a finite number of elements from a set. Arrangements, combinations, and permutations are the three main types of configurations and are applicable to the probability. The probability of an event is, therefore, the ratio between the number of results of the event we are interested in and the total number of results in the sample space (total number of arrangements, combinations, or permutations).
An arrangement calculates the number of possible configurations with distinct elements from a certain set. Bruni (2011) defines arrangement as the study of the number of ways in which researcher can organize a sample of objects, which was removed from a larger population, and in which the alteration of the order of the organized objects is relevant.
Given n different objects, if the objective is to select p of these objects (n and p are integers, n ≥ p), the number of arrangements or possible ways of doing this is represented by An,p and calculated as follows:
Combinations are a special case of arrangements in which it does not matter the order in which the elements are organized.
Given n different objects, the number of ways or combinations in which to organize p of these objects is represented by Cn,p (a combination of n elements arranged p by p), and calculated as follows:
Permutation is an arrangement in which all the elements in the set are selected. Therefore, it is the number of ways in which n elements can be grouped, changing their order. The number of possible permutations is represented by Pn and can be calculated as follows:
This chapter discussed the concepts and terminologies related to the probability theory, as well as their practical application. Probability theory is used to assess the possibility of uncertain events happening, its origin comes from trying to understand uncertain natural phenomena, evolving to planning how to gamble, and, currently, it is being applied to the study of statistical inference.