Title Page Copyright and Credits Learning Elastic Stack 7.0 Second Edition About Packt Why subscribe? Packt.com Contributors About the authors About the reviewer Packt is searching for authors like you Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Reviews Section 1: Introduction to Elastic Stack and Elasticsearch Introducing Elastic Stack What is Elasticsearch, and why use it? Schemaless and document-oriented Searching capability Analytics Rich client library support and the REST API Easy to operate and easy to scale  Near real-time capable Lightning–fast Fault-tolerant Exploring the components of the Elastic Stack Elasticsearch Logstash Beats Kibana X-Pack Security Monitoring Reporting Alerting Graph Machine learning Elastic Cloud Use cases of Elastic Stack Log and security analytics Product search Metrics analytics Web search and website search Downloading and installing Installing Elasticsearch Installing Kibana Summary Getting Started with Elasticsearch Using the Kibana Console UI Core concepts of Elasticsearch Indexes Types Documents Nodes Clusters Shards and replicas Mappings and datatypes Datatypes Core datatypes Complex datatypes Other datatypes Mappings Creating an index with the name catalog Defining the mappings for the type of product Inverted indexes CRUD operations Index API Indexing a document by providing an ID Indexing a document without providing an ID Get API Update API Delete API Creating indexes and taking control of mapping Creating an index Creating type mapping in an existing index Updating a mapping REST API overview Common API conventions Formatting the JSON response Dealing with multiple indexes Searching all documents in one index Searching all documents in multiple indexes Searching all the documents of a particular type in all indexes Summary Section 2: Analytics and Visualizing Data Searching - What is Relevant The basics of text analysis Understanding Elasticsearch analyzers Character filters Tokenizer Standard tokenizer Token filters Using built-in analyzers Standard analyzer Implementing autocomplete with a custom analyzer Searching from structured data Range query Range query on numeric types Range query with score boosting Range query on dates Exists query Term query Searching from the full text Match query Operator Minimum should match Fuzziness Match phrase query Multi match query Querying multiple fields with defaults Boosting one or more fields With types of multi match queries Writing compound queries Constant score query Bool query Combining OR conditions Combining AND and OR conditions Adding NOT conditions Modeling relationships has_child query has_parent query parent_id query Summary Analytics with Elasticsearch The basics of aggregations Bucket aggregations Metric aggregations Matrix aggregations Pipeline aggregations Preparing data for analysis Understanding the structure of the data Loading the data using Logstash Metric aggregations Sum, average, min, and max aggregations Sum aggregation Average aggregation Min aggregation Max aggregation Stats and extended stats aggregations Stats aggregation Extended stats aggregation Cardinality aggregation Bucket aggregations Bucketing on string data Terms aggregation Bucketing on numerical data Histogram aggregation Range aggregation Aggregations on filtered data Nesting aggregations Bucketing on custom conditions Filter aggregation Filters aggregation Bucketing on date/time data Date Histogram aggregation Creating buckets across time periods Using a different time zone Computing other metrics within sliced time intervals Focusing on a specific day and changing intervals Bucketing on geospatial data Geodistance aggregation GeoHash grid aggregation Pipeline aggregations Calculating the cumulative sum of usage over time Summary Analyzing Log Data Log analysis challenges Using Logstash Installation and configuration Prerequisites Downloading and installing Logstash Installing on Windows Installing on Linux Running Logstash The Logstash architecture Overview of Logstash plugins Installing or updating plugins Input plugins Output plugins Filter plugins Codec plugins Exploring plugins Exploring input plugins File Beats JDBC IMAP Output plugins Elasticsearch CSV Kafka PagerDuty Codec plugins JSON Rubydebug  Multiline Filter plugins Ingest node Defining a pipeline  Ingest APIs Put pipeline API Get pipeline API Delete pipeline API Simulate pipeline API Summary Building Data Pipelines with Logstash Parsing and enriching logs using Logstash Filter plugins CSV filter  Mutate filter Grok filter Date filter Geoip filter Useragent filter Introducing Beats Beats by Elastic.co Filebeat Metricbeat Packetbeat Heartbeat Winlogbeat Auditbeat Journalbeat Functionbeat Community Beats Logstash versus Beats Filebeat Downloading and installing Filebeat Installing on Windows Installing on Linux Architecture Configuring Filebeat Filebeat inputs Filebeat general/global options Output configuration  Logging Filebeat modules Summary Visualizing Data with Kibana Downloading and installing Kibana Installing on Windows Installing on Linux Configuring Kibana Preparing data Kibana UI User interaction Configuring the index pattern Discover Elasticsearch query string/Lucene query Elasticsearch DSL query KQL Visualize Kibana aggregations Bucket aggregations Metric Creating a visualization Visualization types Line, area, and bar charts Data tables Markdown widgets Metrics Goals Gauges Pie charts Co-ordinate maps Region maps Tag clouds Visualizations in action Response codes over time Top 10 requested URLs Bandwidth usage of the top five countries over time Web traffic originating from different countries Most used user agent Dashboards Creating a dashboard Saving the dashboard  Cloning the dashboard Sharing the dashboard  Timelion Timelion  Timelion expressions Using plugins Installing plugins Removing plugins Summary Section 3: Elastic Stack Extensions Elastic X-Pack Installing Elasticsearch and Kibana with X-Pack Installation Activating X-Pack trial account Generating passwords for default users Configuring X-Pack Securing Elasticsearch and Kibana User authentication User authorization Security in action Creating a new user Deleting a user Changing the password Creating a new role Deleting or editing a role Document-level security or field-level security X-Pack security APIs User Management APIs Role Management APIs Monitoring Elasticsearch Monitoring UI Elasticsearch metrics Overview tab Nodes tab The Indices tab Alerting Anatomy of a watch Alerting in action Creating a new alert Threshold Alert Advanced Watch Deleting/deactivating/editing a watch Summary Section 4: Production and Server Infrastructure Running Elastic Stack in Production Hosting Elastic Stack on a managed cloud Getting up and running on Elastic Cloud Using Kibana Overriding configuration  Recovering from a snapshot Hosting Elastic Stack on your own Selecting hardware Selecting an operating system Configuring Elasticsearch nodes JVM heap size Disable swapping File descriptors Thread pools and garbage collector Managing and monitoring Elasticsearch Running in Docker containers Special considerations while deploying to a cloud Choosing instance type Changing default ports; do not expose ports! Proxy requests Binding HTTP to local addresses Installing EC2 discovery plugin Installing the S3 repository plugin Setting up periodic snapshots Backing up and restoring Setting up a repository for snapshots Shared filesystem Cloud or distributed filesystems Taking snapshots Restoring a specific snapshot Setting up index aliases Understanding index aliases How index aliases can help Setting up index templates Defining an index template Creating indexes on the fly Modeling time series data Scaling the index with unpredictable volume over time Unit of parallelism in Elasticsearch The effect of the number of shards on the relevance score The effect of the number of shards on the accuracy of aggregations Changing the mapping over time New fields get added Existing fields get removed Automatically deleting older documents How index-per-timeframe solves these issues Scaling with index-per-timeframe Changing the mapping over time Automatically deleting older documents Summary Building a Sensor Data Analytics Application Introduction to the application Understanding the sensor-generated data Understanding the sensor metadata Understanding the final stored data Modeling data in Elasticsearch Defining an index template Understanding the mapping Setting up the metadata database Building the Logstash data pipeline Accepting JSON requests over the web Enriching the JSON with the metadata we have in the MySQL database The jdbc_streaming plugin  The mutate plugin Moving the looked-up fields that are under lookupResult directly in JSON Combining the latitude and longitude fields under lookupResult as a location field Removing the unnecessary fields Store the resulting documents in Elasticsearch Sending data to Logstash over HTTP Visualizing the data in Kibana Setting up an index pattern in Kibana Building visualizations How does the average temperature change over time? How does the average humidity change over time? How do temperature and humidity change at each location over time? Can I visualize temperature and humidity over a map? How are the sensors distributed across departments? Creating a dashboard Summary Monitoring Server Infrastructure Metricbeat Downloading and installing Metricbeat Installing on Windows Installing on Linux Architecture Event structure Configuring Metricbeat Module configuration Enabling module configs in the modules.d directory Enabling module configs in the metricbeat.yml file General settings Output configuration  Logging Capturing system metrics Running Metricbeat with the system module Specifying aliases Visualizing system metrics using Kibana Deployment architecture Summary Other Books You May Enjoy Leave a review - let other readers know what you think