In the previous chapter, we went over the basics of probability and how we can apply simple theorems to complex tasks. To briefly summarize, probability is the mathematics of modeling events that may or may not occur. We use formulas in order to describe these events and even look at how multiple events can behave together. In this chapter, we will explore more complicated theorems of probability and how we can use them in a predictive capacity. Advanced topics, such as Bayes' theorem and random variables, give rise to common machine learning algorithms, such as the Naïve Bayes algorithm (also covered in this book). This chapter will focus on some of the more advanced topics in probability theory, including the following topics:
- Exhaustive events
- Bayes' theorem
- Basic prediction rules
- Random variables
We have one more definition to look at before we get started (the last one before the fun stuff, I promise). We have to look at collectively exhaustive events.
When given a set of two or more events, if at least one of the events must occur, then such a set of events is said to be collectively exhaustive. Consider the following examples:
- Given a set of events
{temperature < 60, temperature > 90}, these events are not collectively exhaustive because there is a third option that is not given in this set of events; the temperature could be between60and90. However, they are mutually exhaustive because both cannot happen at the same time. - In a dice roll, the set of events of rolling a
{1, 2, 3, 4, 5, or 6}are collectively exhaustive because these are the only possible events, and at least one of them must happen.