Log and security analytics

The Elasticsearch, Logstash, and Kibana trio was, previously, very popular as a stack. The presence of Elasticsearch, Logstash, and Kibana (also known as ELK) makes the Elastic Stack an excellent stack for aggregating and analyzing logs in a central place.

Application support teams face a great challenge in administering and managing large numbers of applications deployed across tens or hundreds of servers. The application infrastructure could have the following components:

  • Web servers
  • Application servers
  • Database servers
  • Message brokers

Typically, enterprise applications have all, or most, of the types of servers described earlier, and there are multiple instances of each server. In the event of an error or production issue, the support team has to log in to individual servers and look at the errors. It is quite inefficient to log in to individual servers and look at the raw log files. The Elastic Stack provides a complete toolset to collect, centralize, analyze, visualize, alert, and report errors as they occur. Each component can be used to solve this problem as follows:

  • The Beats framework, Filebeat in particular, can run as a lightweight agent to collect and forward logs.
  • Logstash can centralize events received from Beats, and parse and transform each log entry before sending it to the Elasticsearch cluster.
  • Elasticsearch indexes logs. It enables both search and analytics on the parsed logs.
  • Kibana then lets you create visualizations based on errors, warnings, and other information logs. It lets you create dashboards on which you can centrally monitor events as they occur, in real time.
  • With X-Pack, you can secure the solution, configure alerts, get reports, and analyze relationships in data.

As you can see, you can get a complete log aggregation and monitoring solution using Elastic Stack.

A security analytics solution would be very similar to this; the logs and events being fed into the system would pertain to firewalls, switches, and other key network elements.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset