As shown in the previous chapter, Chapter 2, Installing the Elastic Stack with Machine Learning, tracking metrics and their potential abnormalities over time is certainly an extremely important application of anomaly detection to IT data. This affords a broad, proactive coverage of many key indicators of performance and availability.

However, there are many important use cases that revolve around the idea of event change detection. These include the following:

• Discovering a flood of error messages suddenly cropping up in a log file
• Detecting a sudden drop in the amount of orders processed by an online system
• Determining a sudden excessive number of attempts at accessing something (for example, brute-force authentication or reconnaissance scanning)

In this chapter, we'll discuss the concepts of determining anomalies based on the occurrence rates of things, and we will go through several practical examples such as the following:

• Count functions
• Counting in population analysis
• Detecting things that rarely occur
• Counting message-based logs via categorization