6.2.1 Discrete Random Variable

A discrete random variable can only take on countable numbers of distinct values, usually counts. Therefore, it cannot assume decimal or noninteger values. As examples of discrete random variables, we can mention the number of children in a family, the number of employees in a company, or the number of vehicles produced in a certain factory.

6.2.2 Continuous Random Variable

A continuous random variable can take on several different values in an interval of real numbers. As examples of continuous random variables, we can mention a family’s income, the revenue of a company, or the height of a certain child.

A continuous random variable X is associated to an f(x) function, called a probability density function (p.d.f.) of X, which meets the following condition:

$\underset{-\infty }{\overset{+\infty }{\int }}f\left(x\right)\mathit{dx}=1,f\left(x\right)\ge 0$

For any a and b, such that − ∞ < a < b < + ∞, the probability of random variable X taking on values within this interval is:

$P\left(a\le X\le b\right)=\underset{a}{\overset{b}{\int }}f\left(x\right)\mathit{dx}$

which can be graphically represented as shown in Fig. 6.1.

6.3.1 Discrete Uniform Distribution

It is the simplest discrete probability distribution and receives the name uniform because all of the possible values of the random variable have the same probability of occurrence.

A discrete random variable X that takes on the values x₁, x₂, …, x_n has a discrete uniform distribution with parameter n, denoted by X ~ U_d{x₁, x₂, …, x_n}, if its probability function is given by:

$P\left(X=x_i\right)=p\left(x_i\right)=\frac{1}{n},i=1,2,\dots ,n$

which may be graphically represented as shown in Fig. 6.2.

The mathematical expected value of X is given by:

$E\left(X\right)=\frac{1}{n}.\sum _{i=1}^n{x}_i$

The variance of X is calculated as:

$\mathit{Var}\left(X\right)=\frac{1}{n}.\left[\sum _{i=1}^n{x}_i^2-\frac{{\left(\sum _{i=1}^n{x}_i\right)}^2}{n}\right]$

And the cumulative distribution function (c.d.f.) is:

$F\left(X\right)=P\left(X\le x\right)=\sum _{x_i\le x}\frac{1}{n}=\frac{n\left(x\right)}{n},$

where n(x) is the number of x_i ≤ x, as shown in Fig. 6.3.

6.3.2 Bernoulli Distribution

The Bernoulli trial is a random experiment that only offers two possible results, conventionally called success or failure. As an example of a Bernoulli trial, we can mention tossing a coin, whose only possible results are head and tail.

For a certain Bernoulli trial, we will consider the random variable X that takes on the value 1 in case of success, and 0 in case of failure. The probability of success is represented by p and the probability of failure by (1 − p) or q. The Bernoulli distribution, therefore, provides the probability of success or failure of variable X when carrying out a single experiment. Therefore, we can say that variable X follows a Bernoulli distribution with parameter p, denoted by X ~ Bern(p), if its probability function is given by:

$P\left(X=x\right)=p\left(x\right)=\left\{\begin{array}{l}q=1-p,\mathrm{if}x=0\\ p,\mathrm{if}x=1\end{array}\right.$

which can also be represented in the following way:

$P\left(X=x\right)=p\left(x\right)=p^x.{\left(1-p\right)}^{1-x},x=0,1$

The probability function of random variable X is represented in Fig. 6.4.

It is easy to see that the expected value of X is:

$E\left(X\right)=p$

with X’s variance being:

$\mathit{Var}\left(X\right)=p.\left(1-p\right)$

Bernoulli’s cumulative distribution function (c.d.f.) is given by:

$F\left(x\right)=P\left(X\le x\right)=\left\{\begin{array}{l}0,\mathrm{if}x<0\\ 1-p,\mathrm{if}x\le 0<1\\ 1,\mathrm{if}x\ge 1\end{array}\right.$

which can be represented by Fig. 6.5.

It is important to mention that we are going to use all knowledge on Bernoulli’s distribution when discussing binary logistics regression models (Chapter 14).

6.3.3 Binomial Distribution

A binomial experiment consists in n independent repetitions of a Bernoulli trial with probability of success p, probability that remains constant in all repetitions.

Discrete random variable X of a binomial model corresponds to the number of successes (k) in the n repetitions of the experiment. Therefore, X follows a binomial distribution with parameters n and p, denoted by X ~ b(n, p), if its probability distribution function is given by:

$f\left(k\right)=P\left(X=k\right)=\left(\begin{array}{l}n\\ k\end{array}\right).p^k.{\left(1-p\right)}^{n-k},k=0,1,\dots ,n$

where $\left(\begin{array}{l}n\\ k\end{array}\right)=\frac{n!}{k!\left(n-k\right)!}$

The mean of X is given by:

$E\left(X\right)=n.p$

On the other hand, the variance of X can be expressed as:

$\mathit{Var}\left(X\right)=n.p.\left(1-p\right)$

Note that the mean and variance of the binomial distribution are equal to the mean and variance of the Bernoulli distribution, multiplied by n, the number of repetitions in a Bernoulli trial.

Fig. 6.6 shows the probability function of the binomial distribution for n = 10 and p varying between 0.3, 0.5, and 0.7.

From Fig. 6.6, we can see that, for p = 0.5, the probability function is symmetrical around the mean. If p < 0.5, the distribution is positive asymmetrical, observing a higher frequency of smaller values of k and a longer tail to the right. If p > 0.5, the distribution is negative asymmetrical, observing a higher frequency of larger values of k and a longer tail to the left.

It is important to mention that we are going to use all knowledge on the binomial distribution when studying multinomial logistics regression models (Chapter 14).

6.3.4 Geometric Distribution

The geometric distribution, as well as the binomial, considers successive independent Bernoulli trials, all of them with probability of success p. However, instead of using a fixed number of trials, they will carry out the experiment until the first success is obtained. The geometric distribution presents two distinct parameterizations, described here.

The first parameterization considers successive independent Bernoulli trials, with probability of success p in each trial, until a success occurs. In this case, we cannot include zero as a possible result, so, the domain is supported by using the set {1, 2, 3, …}. For example, we can consider how many times we tossed a coin until we got the first head, the number of parts manufactured until a defective one was produced, among others.

The second parameterization of the geometric distribution counts the number of failures or unsuccessful attempts before the first success. Since here it is possible to obtain success in the first Bernoulli trial, we include zero as being a possible result, so, the domain is supported by the set {0, 1, 2, 3, …}.

Let X be the random variable that represents the number of trials until the first success. Variable X has a geometric distribution with parameter p, denoted by X ~ Geo(p), if its probability function is given by:

$f\left(x\right)=P\left(X=x\right)=p.{\left(1-p\right)}^{x-1},x=1,2,3,\dots$

For the second case, let us consider Y the random variable that represents the number of failures or unsuccessful attempts before the first success. Variable Y has a geometric distribution with parameter p, denoted by Y ~ Geo(p), if its probability function is given by:

$f\left(y\right)=P\left(Y=y\right)=p.{\left(1-p\right)}^y,y=0,1,2,\dots$

In both cases, the sequence of probabilities is a geometric progression.

The probability function of variable X is graphically represented in Fig. 6.7, for p = 0.4.

The calculation of X’s expected value and variance is:

$E\left(X\right)=\frac{1}{p}$

$\mathit{Var}\left(X\right)=\frac{1-p}{p^2}$

In a similar way, for variable Y, we have:

$E\left(Y\right)=\frac{1-p}{p}$

$\mathit{Var}\left(Y\right)=\frac{1-p}{p^2}$

The geometric distribution is the only discrete distribution that has the memoryless property (in the case of continuous distributions, we will see that the exponential distribution also has this property). This means that if an experiment is repeated before the first success, then, given that the first success has not happened yet, the conditional distribution function of the number of additional trials does not depend on the number of failures that occurred until then.

Thus, for any two positive integers s and t, if X is greater than s, then, the probability of X being greater than s + t is equal to the unconditional probability of X being greater than t:

$P\left(X>s+t\left|X>s\right.\right)=P\left(X>t\right)$

6.3.5 Negative Binomial Distribution

The negative binomial distribution, also known as the Pascal distribution, carries out successive independent Bernoulli trials (with a constant probability of success in all the trials) until it achieves a prefixed number of successes (k), that is, the experiment continues until k successes are achieved.

Let X be the random variable that represents the number of attempts carried out (Bernoulli trials) until the k-th success is reached. Variable X has a negative binomial distribution, denoted by X ~ nb(k, p), if its probability function is given by:

$f\left(x\right)=P\left(X=x\right)=\left(\begin{array}{l}x-1\\ k-1\end{array}\right).p^k.{\left(1-p\right)}^{x-k},x=k,k+1,\dots$

The graphical representation of a negative binomial distribution with parameter k = 2 and p = 0.4 can be found in Fig. 6.8.

The expected value of X is:

$E\left(X\right)=\frac{k}{p}$

and the variance is:

$\mathit{Var}\left(X\right)=\frac{k.\left(1-p\right)}{p^2}$

6.3.6 Hypergeometric Distribution

The hypergeometric distribution is also related to a Bernoulli trial. However, differently from the binomial sampling, in which the probability of success is constant, in the hypergeometric distribution, since the sampling is without replacement, as the elements are removed from the population to form the sample, the population size diminishes, making the probability of success vary.

The hypergeometric distribution describes the number of successes in a sample with n elements, drawn from a finite population without replacement. For example, let us consider a population with N elements, from which M have a certain attribute. The hypergeometric distribution describes the probability of exactly k elements having such attribute (k successes and n − k failures), in a sample with n distinct elements randomly drawn from the population without replacement.

Let X be a random variable that represents the number of successes obtained from the n elements drawn from the sample. Variable X follows a hypergeometric distribution with parameters N, M, n, denoted by X ~ Hip(N, M, n), if its probability function is given by:

$f\left(k\right)=P\left(X=k\right)=\frac{\left(\begin{array}{l}M\\ k\end{array}\right).\left(\begin{array}{l}N-M\\ n-k\end{array}\right)}{\left(\begin{array}{l}N\\ n\end{array}\right)},0\le k\le min\left(M,n\right)$

The graphical representation of a hypergeometric distribution with parameters N = 200, M = 50, and n = 30 can be found in Fig. 6.9.

The mean of X can be calculated as:

$E\left(X\right)=\frac{n.M}{N}$

with variance:

$\mathit{Var}\left(X\right)=\frac{n.M}{N}.\frac{\left(N-M\right).\left(N-n\right)}{N.\left(N-1\right)}$

6.3.7 Poisson Distribution

The Poisson distribution is used to register the occurrence of rare events, with a very low probability of success (p → 0), in a certain time interval or space.

Differently from the binomial model that provides the probability of the number of successes in a discrete interval (n repetitions of an experiment), the Poisson model provides the probability of the number of successes in a certain continuous interval (time, area, among other possibilities). As examples of variables that represent a Poisson distribution, we can mention the number of customers that arrive in a line per unit of time, the number of defects per unit of time, the number of accidents per unit of area, among others. Note that the measurement units (time and area, in these situations) are continuous, but the random variable (number of occurrences) is discrete.

The Poisson distribution presents the following hypotheses:

1. (i) Events defined in nonoverlapping intervals are independent;
2. (ii) In intervals with the same length, the probabilities that the same number of successes will occur are equal;
3. (iii) In very small intervals, the probability that more than one success will occur is insignificant;
4. (iv) In very small intervals, the probability of one success is proportional to the length of the interval.

Let us consider a discrete random variable X that represents the number of successes (k) in a certain unit of time, unit of area, among other possibilities. Random variable X, with parameter λ ≥ 0, follows a Poisson distribution, denoted by X ~ Poisson (λ), if its probability function is given by:

$f\left(k\right)=P\left(X=k\right)=\frac{e^{-\lambda }.{\lambda }^k}{k!},k=0,1,2,\dots$

where:

• e: base of the Napierian (or natural) logarithm, and e ≅ 2.718282;
• λ: estimated average rate of occurrence of the event we are interested in for a certain exposition (time interval, area, among other examples).

Fig. 6.10 shows the Poisson distribution probability function for λ = 1, 3, and 6.

In the Poisson distribution, the mean is equal to the variance:

$E\left(X\right)=\mathit{VAR}\left(X\right)=\lambda$

It is important to mention that we are going to use all knowledge on the Poisson distribution when studying the regression models for count data (Chapter 15).

6.4.1 Uniform Distribution

The uniform model is the simplest model for continuous random variables. It is used to model the occurrence of events whose probability is constant in intervals with the same range.

A random variable X follows a uniform distribution in the interval [a, b], denoted by X ~ U[a, b], if its probability density function is given by:

$f\left(x\right)=\left\{\begin{array}{l}1/\left(b-a\right),\mathrm{if}a\le x\le b\\ \mathtt{0},\mathrm{otherwise}\end{array}\right.$

which can be graphically represented as seen in Fig. 6.11.

The expected value of X is calculated by the expression:

$E\left(X\right)=\underset{a}{\overset{b}{\int }}x\frac{1}{b-a}\mathit{dx}=\frac{a+b}{2}$

Table 6.1 presents a summary of the discrete distributions studied in this section, including the calculation of the random variable's probability function, the distribution parameters, besides the calculation of X’s expected value and variance.

Table 6.1

Models for Discrete Variables

Distribution	Probability Function – P(X)	Parameters	E(X)	Var(X)
Discrete uniform	$\frac{1}{n}$	n	$\frac{1}{n}.\sum _{i=1}^n{x}_i$	$\frac{1}{n}.\left[\sum _{i=1}^n{x}_i^2-\frac{{\left(\sum _{i=1}^n{x}_i\right)}^2}{n}\right]$
Bernoulli	px. (1 − p)1 − x, x = 0, 1	p	p	p. (1 − p)
Binomial	$\left(\begin{array}{l}n\\ k\end{array}\right).p^k.{\left(1-p\right)}^{n-k},k=0,1,\dots ,n$	n, p	n.p	n. p. (1 − p)
Geometric	P(X) = p. (1 − p)x − 1, x = 1, 2, 3, … P(Y) = p. (1 − p)y, y = 0, 1, 2, …	p	$E\left(X\right)=\frac{1}{p}$ $E\left(Y\right)=\frac{1-p}{p}$	$$\begin{gathered} Var\left(X\right)=\frac{1-p}{p^2} \\ Var\left(Y\right)=\frac{1-p}{p^2} \end{gathered}$$
Negative binomial	$\left(\begin{array}{l}x-1\\ k-1\end{array}\right).p^k.{\left(1-p\right)}^{x-k},x=k,k+1,\dots$	k, p	$\frac{k}{p}$	$\frac{k.\left(1-p\right)}{p^2}$
Hypergeometric	$\frac{\left(\begin{array}{l}M\\ k\end{array}\right).\left(\begin{array}{l}N-M\\ n-k\end{array}\right)}{\left(\begin{array}{l}N\\ n\end{array}\right)},0\le k\le min\left(M,n\right)$	N, M, n	$\frac{n.M}{N}$	$\frac{n.M}{N}.\frac{\left(N-M\right).\left(N-n\right)}{N.\left(N-1\right)}$
Poisson	$\frac{e^{-\lambda }.{\lambda }^k}{k!},k=0,1,2,\dots$	λ	λ	λ

And the variance of X is:

$\mathit{Var}\left(X\right)=E\left(X^2\right)-{\left[E\left(X\right)\right]}^2=\frac{{\left(b-a\right)}^2}{12}$

On the other hand, the cumulative distribution function of the uniform distribution is given by:

$F\left(x\right)=P\left(X\le x\right)=\underset{a}{\overset{x}{\int }}f\left(x\right)dx=\underset{a}{\overset{x}{\int }}\frac{1}{b-a}dx=\left\{\begin{array}{l}0,\mathrm{if}x<a\\ \frac{x-a}{b-a},\mathrm{if}a\le x<b\\ 1,\mathrm{if}x\ge \mathrm{b}\end{array}\right.$

6.4.2 Normal Distribution

The normal distribution, also known as Gaussian, is the most widely used and important probability distribution, because it allows us to model a myriad of natural phenomena, studies of human behavior, industrial processes, among others. In addition to allowing us to use approximations to calculate the probabilities of many random variables.

A random variable X, with mean μ ∈ ℜ and standard deviation σ > 0, follows a normal or Gaussian distribution, denoted by X ~ N (μ, σ²), if its probability density function is given by:

$f\left(x\right)=\frac{1}{\sigma .\sqrt{2\pi }}.e^{-\frac{{\left(x-\mu \right)}^2}{2.{\sigma }^2}},-\infty \le x\le +\infty ,$

whose graphical representation is shown in Fig. 6.12.

Fig. 6.13 shows the area under the normal curve based on the number of standard deviations.

From Fig. 6.13, we can see that the curve has the shape of a bell and is symmetrical around parameter μ, and the smaller parameter σ is, the more concentrated the curve is around μ.

Therefore, in a normal distribution, the mean of X is:

$E\left(X\right)=\mu$

And the variance of X is:

$\mathit{Var}\left(X\right)={\sigma }^2$

In order to obtain the standard normal distribution or the reduced normal distribution, the original variable X is transformed into a new random variable Z, with mean 0 (μ = 0) and variance 1 (σ² = 1):

$Z=\frac{X-\mu }{\sigma }~N\left(0,1\right)$

Score Z represents the number of standard deviations that separates a random variable X from the mean.

This kind of transformation, known as Zscores, is broadly used to standardize variables, because it does not change the shape of the original variable's normal distribution, and it generates a new variable with mean 0 and variance 1. Therefore, when many variables with different orders of magnitude are being used in a certain type of modeling, the Zscores standardization process will make all the new standardized variables have the same distribution, with equal orders of magnitude (Fávero et al., 2009).

The probability density function of random variable Z is reduced to:

$f\left(z\right)=\frac{1}{\sqrt{2\pi }}.e^{\frac{-z^2}{2}},-\infty \le z\le +\infty$

whose graphical representation is shown in Fig. 6.14.

The cumulative distribution function F(x_c) of a normal random variable X is obtained by integrating Expression (6.51) from −∞ to x_c, that is:

$F\left(x_c\right)=P\left(X\le x_c\right)=\underset{-\infty }{\overset{x_c}{\int }}f\left(x\right)\mathit{dx}$

Integral (6.56) corresponds to the area under f(x) from −∞ to x_c, as shown in Fig. 6.15.

In the specific case of the standard normal distribution, the cumulative distribution function is:

$F\left(z_c\right)=P\left(Z\le z_c\right)=\underset{-\infty }{\overset{z_c}{\int }}f\left(z\right)\mathit{dz}=\frac{1}{\sqrt{2\pi }}\underset{-\infty }{\overset{z_c}{\int }}{e}^{-\frac{z^2}{2}\mathit{dz}}$

For a random variable Z with a standard normal distribution, let us suppose that the main goal now is to calculate P(Z > z_c). So, we have:

$P\left(Z>z_c\right)=\underset{z_c}{\overset{\infty }{\int }}f\left(z\right)\mathit{dz}=\frac{1}{\sqrt{2\pi }}\underset{z_c}{\overset{\infty }{\int }}{e}^{\frac{-z^2}{2}\mathit{dz}}$

Fig. 6.16 represents this probability.

Table E in the Appendix shows the value of P(Z > z_c), that is, the cumulative probability from z_c to +∞ (the gray area under the normal curve).

6.4.3 Exponential Distribution

Another important distribution, which has applications in system reliability and in the queueing theory, is the exponential distribution. It has as its main characteristic the property of being memoryless, that is, the future lifetime (t) of a certain object has the same distribution, regardless of its past lifetime (s), for any s, t > 0, as shown in Expression (6.38), once again shown below:

$P\left(X>s+t\left|X>s\right.\right)=P\left(X>t\right)$

A continuous random variable X has an exponential distribution with parameter λ > 0, denoted by X ~ exp(λ), if its probability density function is given by:

$f\left(x\right)=\left\{\begin{array}{l}\lambda .e^{-\lambda .x},\mathrm{if}x\ge 0\\ 0,\mathrm{if}x<0\end{array}\right.$

Fig. 6.17 represents the probability density function of the exponential distribution for parameters λ = 0.5, λ = 1, and λ = 2.

We can see that the exponential distribution is positive asymmetrical (to the right), observing a higher frequency for smaller values of x and a longer tail to the right. The density function assumes value λ when x = 0, and tends to zero as x → ∞. The higher the value of λ, the more quickly the function tends to zero.

In the exponential distribution, the mean of X is:

$E\left(X\right)=\frac{1}{\lambda }$

and the variance of X is:

$\mathit{Var}\left(X\right)=\frac{1}{{\lambda }^2}$

And the cumulative distribution function F(x) is given by:

$F\left(x\right)=P\left(X\le x\right)=\underset{0}{\overset{x}{\int }}f\left(x\right)\mathit{dx}=\left\{\begin{array}{l}1-e^{-\lambda .x},\mathrm{if}x\ge 0\\ 0,\mathrm{if}x<0\end{array}\right.$

From (6.62) we can conclude that:

$P\left(X>x\right)=e^{-\lambda .x}$

In system reliability, random variable X represents the lifetime, that is, the time during which a component or system remains operational, outside the interval for repairs and above a specified limit (yield, pressure, among other examples). On the other hand, parameter λ represents the failure rate, that is, the number of components or systems that failed in a pre-established time interval:

$\lambda =\frac{\text{number of failures}}{\text{operation time}}$

The main measures of reliability are: (a) Mean time to failure (MTTF) and (b) Mean time between failures (MTBF). Mathematically, MTTF and MTBF are equal to the mean of the exponential distribution and represent the mean lifetime. Thus, the failure rate can also be calculated as:

$\lambda =\frac{1}{\text{MTTF}.\left(\text{MTBF}\right)}$

In the queueing theory, random variable X represents the mean waiting time until the next arrival (mean time between two customers’ arrivals). On the other hand, parameter λ represents the mean arrivals rate, that is, the expected number of arrivals per unit of time.

6.4.4 Gamma Distribution

The gamma distribution is one of the most general, such that, other distributions, as the Erlang, exponential, and chi-square (χ²) are particular cases of it. As the exponential distribution, it is also widely used in system reliability. The gamma distribution also has applications in physical phenomena, in meteorological processes, insurance risk theory, and economic theory.

A continuous random variable X has a gamma distribution with parameters α > 0 and λ > 0, denoted by X ~ Gamma(α, λ), if its probability density function is given by:

$f\left(x\right)=\left\{\begin{array}{l}\frac{{\lambda }^{\alpha }}{\mathrm{\Gamma }\left(\alpha \right)}.x^{\alpha -1}.e^{-\lambda .x},\mathrm{if}x\ge 0\\ 0,\mathrm{if}x<0\end{array}\right.$

where Γ(α) is the Gamma function, given by:

$\mathrm{\Gamma }\left(\alpha \right)=\underset{0}{\overset{\infty }{\int }}{e}^{-x}.x^{\alpha -1}\mathit{dx},\alpha >0$

The gamma probability density function for some values of α and λ is represented in Fig. 6.18.

We can see that the gamma distribution is positive asymmetrical (to the right), observing a higher frequency for smaller values of x and a longer tail to the right. However, as α tends to infinity, the distribution becomes symmetrical. We can also observe that when α = 1, the gamma distribution is equal to the exponential. Moreover, the greater the value of λ, the more quickly the density function tends to zero.

The expected value of X can be calculated as:

$E\left(X\right)=\alpha .\lambda$

On the other hand, the variance of X is given by:

$\mathit{Var}\left(X\right)=\alpha .{\lambda }^2$

The cumulative distribution function is:

$F\left(x\right)=P\left(X\le x\right)=\underset{0}{\overset{x}{\int }}f\left(x\right)\mathit{dx}=\frac{{\lambda }^{\alpha }}{\mathrm{\Gamma }\left(\alpha \right)}\underset{0}{\overset{x}{\int }}{x}^{\alpha -1}.e^{-\mathit{\lambda x}}\mathit{dx}$

6.4.5 Chi-Square Distribution

A continuous random variable X has a chi-square distribution with ν degrees of freedom, denoted by X ~ χ_ν², if its probability density function is given by:

$f\left(x\right)=\left\{\begin{array}{l}\frac{1}{2^{\nu /2}.\mathrm{\Gamma }\left(\nu /2\right)}.x^{\nu /2-1}.e^{-x/2},x>0\\ 0,x<0\end{array}\right.$

where $\mathrm{\Gamma }\left(\alpha \right)=\underset{0}{\overset{\infty }{\int }}{e}^{-x}.x^{\alpha -1}\mathit{dx}$

The χ² distribution can be simulated from a normal distribution. Consider Z₁, Z₂, …, Z_ν independent random variables with a standard normal distribution (mean 0 and standard deviation 1). So, the sum of the squares of the ν random variables is a chi-square distribution with ν degrees of freedom:

${\chi }_{\nu }^2=Z_1^2+Z_2^2+\dots +Z_{\nu }^2$

The χ² distribution has a positive asymmetrical curve. The graphical representation of the χ² distribution, for different values of ν, is shown in Fig. 6.19.

Since the χ² distribution comes from the sum of the squares of ν random variables that have a normal distribution with mean zero and variance 1, for high values of ν, the χ² distribution becomes more similar to a standard normal distribution, as can be observed in Fig. 6.19 (Fávero et al., 2009). We can also see that the χ² distribution with 2 degrees of freedom is equivalent to an exponential distribution with λ = 1/2.

The expected value of X can be calculated as:

$E\left(X\right)=\nu$

On the other hand, the variance of X is given by:

$\mathit{Var}\left(X\right)=2.\nu$

The cumulative distribution function is:

$F\left(x_c\right)=P\left(X\le x_c\right)=\underset{-\infty }{\overset{x_c}{\int }}f\left(x\right)\mathit{dx}=\frac{\gamma \left(\nu /2,x_c/2\right)}{\mathrm{\Gamma }\left(\nu /2\right)}$

where $\gamma \left(a,x_c\right)=\underset{0}{\overset{x_c}{\int }}{x}^{a-1}.e^{-x}\mathit{dx}$

If the main goal is to calculate P(X > x_c), we have:

$P\left(X>x_c\right)=\underset{x_c}{\overset{\infty }{\int }}f\left(x\right)\mathit{dx}$

which can be represented by Fig. 6.20.

The χ² distribution has several applications in statistical inference. Due to its importance, the χ² distribution can be found in Table D in the Appendix, for different values of parameter ν. This table provides the critical values of x_c such that P(X > x_c) = α. In other words, we can obtain the calculation of the probabilities and of the cumulative probability density function for different values of x from random variable X.

6.4.6 Student’s t Distribution

Student’s t distribution was developed by William Sealy Gosset, and it is one of the main probability distributions, with several applications in statistical inference.

We are going to assume a random variable Z that has a normal distribution with mean 0 and standard deviation 1, and a random variable X with a chi-square distribution with ν degrees of freedom, such that, Z and X are independent. Continuous random variable T is then defined as:

$T=\frac{Z}{\sqrt{X/\nu }}$

We can say that variable T follows Student’s t distribution with ν degrees of freedom, denoted by T ~ t_ν, if its probability density function is given by:

$f\left(t\right)=\frac{\mathrm{\Gamma }\left(\frac{\nu +1}{2}\right)}{\mathrm{\Gamma }\left(\frac{\nu }{2}\right).\sqrt{\mathit{\pi \nu }}}\cdot {\left(1+\frac{t^2}{\nu }\right)}^{-\frac{\nu +1}{2}},-\infty <t<\infty$

where $\mathrm{\Gamma }\left(\alpha \right)=\underset{0}{\overset{\infty }{\int }}{e}^{-x}.x^{\alpha -1}\mathit{dx}$

Fig. 6.21 shows the behavior of Student’s t distribution probability density function for different degrees of freedom ν, in comparison to the standardized normal distribution.

Note that Student’s t distribution is symmetrical around the mean, with the shape of a bell, and is similar to a standard normal distribution. However, it has wider tails, which can generate more extreme values than a normal distribution.

Parameter ν (number of degrees of freedom) defines and characterizes Student’s t distribution’s shape. The greater the value of ν, the more Student’s t distribution gets closer to a standardized normal distribution.

The expected value of T is given by:

$E\left(T\right)=0$

On the other hand, the variance of T can be calculated as:

$\mathit{Var}\left(T\right)=\frac{\nu }{\nu -2},\nu >2$

The cumulative distribution function is:

$F\left(t_c\right)=P\left(T\le t_c\right)=\underset{-\infty }{\overset{t_c}{\int }}f\left(t\right)\mathit{dt}$

If the main goal is to calculate P(T > t_c), we have:

$P\left(T>t_c\right)=\underset{t_c}{\overset{\infty }{\int }}f\left(t\right)\mathit{dt}$

as shown in Fig. 6.22.

Just as the normal and chi-square (χ²) distributions, Student’s t distribution has several applications in statistical inference, such that, there is a table to obtain the probabilities, based on different values of parameter ν (Table B in the Appendix). This table provides the critical values of t_c such that P(T > t_c) = α. In other words, we can obtain the calculation of the probabilities and of the cumulative probability density function for different values of t from random variable T.

We are going to use Student's t distribution when studying simple and multiple regression models (Chapter 13).

6.4.7 Snedecor’s F Distribution

Snedecor's F distribution, also known as Fisher’s distribution, is frequently used in tests associated to the analysis of variance (ANOVA), to compare the means of more than two populations.

Let us consider continuous random variables Y₁ and Y₂, such that:

• • Y₁ and Y₂ are independent;
• • Y₁ follows a chi-square distribution with ν₁ degrees of freedom, denoted by Y₁ ~ χ_ν1²;
• • Y₂ follows a chi-square distribution with ν₂ degrees of freedom, denoted by Y₂ ~ χ_ν2².

We are going to define a new continuous random variable X such that:

$X=\frac{Y_1/{\nu }_1}{Y_2/{\nu }_2}$

So, we say that X has a Snedecor’s F distribution with ν₁ and ν₂ degrees of freedom, denoted by X ~ F_{ν1, ν2}, if its probability density function is given by:

$f\left(x\right)=\frac{\mathrm{\Gamma }\left(\frac{{\nu }_1+{\nu }_2}{2}\right)\cdot {\left(\frac{{\nu }_1}{{\nu }_2}\right)}^{{\nu }_1/2}\cdot x^{\left({\nu }_1/2\right)-1}}{\mathrm{\Gamma }\left(\frac{{\nu }_1}{2}\right)\cdot \mathrm{\Gamma }\left(\frac{{\nu }_2}{2}\right)\cdot {\left[\left(\frac{{\nu }_1}{{\nu }_2}\right).x+1\right]}^{\left({\nu }_1+{\nu }_2\right)/2}},x>0$

where

$\mathrm{\Gamma }\left(\alpha \right)=\underset{0}{\overset{\infty }{\int }}{e}^{-x}.x^{\alpha -1}\mathit{dx}$

Fig. 6.23 shows the behavior of Snedecor’s F distribution probability density function, for different values of ν₁ and ν₂.

We can see that Snedecor's F distribution is positive asymmetrical (to the right), observing a higher frequency for smaller values of x and a longer tail to the right. However, as ν₁ and ν₂ tend to infinity, the distribution becomes symmetrical.

The expected value of X is calculated as:

$E\left(X\right)=\frac{{\nu }_2}{{\nu }_2-2},\text{for}{\nu }_2>2$

On the other hand, the variance of X is given by:

$\mathit{Var}\left(X\right)=\frac{2.{\nu }_2^2.\left({\nu }_1+{\nu }_2-2\right)}{{\nu }_1.\left({\nu }_2-4\right).{\left({\nu }_2-2\right)}^2},\text{for}{\nu }_2>4$

Just as the normal, χ², and Student’s t distributions, Snedecor’s F distribution has several applications in statistical inference. And there is a table from which we can obtain the probabilities and the cumulative distribution function, based on different values of parameters ν₁ and ν₂ (Table A in the Appendix). This table provides the critical values of F_c such that P(X > F_c) = α (Fig. 6.24).

We are going to use Snedecor’s F distribution when studying simple and multiple regression models (Chapter 13).

6.4.7.1 Relationship Between Student’s t and Snedecor’s F Distribution

Let us consider a random variable T with Student's t distribution with ν degrees of freedom. So, the square of variable T follows Snedecor's F distribution with ν₁ = 1 and ν₂ degrees of freedom, as shown by Fávero et al. (2009). Thus:

If T ~ t_ν, then T² ~ F_{1, ν2}

Example 6.18

Assume that random variable X follows Snedecor’s F distribution with ν₁ = 6 degrees of freedom in the numerator, and ν₂ = 12 degrees of freedom in the denominator, that is, X ~ F_{6, 12}. Determine:

1. a) P(X > 3)
2. b) F_{6, 12} with α = 10%
3. c) The x value such that P(X ≤ x) = 0.975

Solution

Through Snedecor’s F distribution table (Table A in the Appendix), for ν₁ = 6 and ν₂ = 12, we have:

1. a) P(X > 3) = 5%
2. b) 2.33
3. c) 3.73

Table 6.2 shows a summary of the continuous distributions studied in this section, including the calculation of the random variable’s probability function, the distribution parameters, besides the calculation of X’s expected value and variance.

Table 6.2

Models for Continuous Variables

Distribution	Probability Function – P(X)	Parameters	E(X)	Var(X)
Uniform	$\frac{1}{b-a},a\le x\le b$	a, b	$\frac{a+b}{2}$	$\frac{{\left(b-a\right)}^2}{12}$
Normal	$\frac{1}{\sigma .\sqrt{2\pi }}.e^{-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}}$, − ∞ ≤ x ≤ + ∞	μ, σ	μ	σ2
Exponential	λ. e− λ. x,   x ≥ 0	λ	$\frac{1}{\lambda }$	$\frac{1}{{\lambda }^2}$
Gamma	$\frac{{\lambda }^{\alpha }}{\mathrm{\Gamma }\left(\alpha \right)}.x^{\alpha -1}.e^{-\mathit{\lambda x}},x\ge 0$	α, λ	α. λ	α. λ2
Chi-square (χ2)	$\frac{1}{2^{\nu /2}.\mathrm{\Gamma }\left(\nu /2\right)}.x^{\nu /2-1}.e^{-x/2},x>0$	ν	ν	2. ν
Student's t	$\frac{\mathrm{\Gamma }\left(\frac{\nu +1}{2}\right)}{\mathrm{\Gamma }\left(\frac{\nu }{2}\right).\sqrt{\mathit{\pi \nu }}}\cdot {\left(1+\frac{t^2}{\nu }\right)}^{-\frac{\nu +1}{2}}$,− ∞ < t < ∞	ν	E(T) = 0	$\mathit{Var}\left(T\right)=\frac{\nu }{\nu -2}$
Snedecor's F	$\frac{\mathrm{\Gamma }\left(\frac{{\nu }_1+{\nu }_2}{2}\right)\cdot {\left(\frac{{\nu }_1}{{\nu }_2}\right)}^{{\nu }_1/2}\cdot x^{\left({\nu }_1/2\right)-1}}{\mathrm{\Gamma }\left(\frac{{\nu }_1}{2}\right)\cdot \mathrm{\Gamma }\left(\frac{{\nu }_2}{2}\right)\cdot {\left[\left(\frac{{\nu }_1}{{\nu }_2}\right).x+1\right]}^{\left({\nu }_1+{\nu }_2\right)/2}}$ x > 0	ν1, ν2	$\frac{{\nu }_2}{{\nu }_2-2}$	$\frac{2.{\nu }_2^2.\left({\nu }_1+{\nu }_2-2\right)}{{\nu }_1.\left({\nu }_2-4\right).{\left({\nu }_2-2\right)}^2}$