This is a powerful and often used plugin for parsing the unstructured data into structured data, thus making the data easily queryable/filterable. In simple terms, Grok is a way of matching a line against a pattern (which is based on a regular expression) and mapping specific parts of the line to dedicated fields. The general syntax of a grok pattern is as follows:

%{PATTERN:FIELDNAME}

PATTERN is the name of the pattern that will match the text. FIELDNAME is the identifier for the piece of text being matched. 

By default, groked fields are strings. To cast either to float or int values, you can use the following format:

%{PATTERN:FIELDNAME:type}

Logstash ships with about 120 patterns by default. These patterns are reusable and extensible. You can create a custom pattern by combining existing patterns. These patterns are based on the Oniguruma regular expression library.

Patterns consist of a label and a regex. For example:

USERNAME [a-zA-Z0-9._-]+

Patterns can contain other patterns, too; for example:

HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}

If a pattern is not available, then you can use a regular expression by using the following format:

(?<field_name>regex)

For example, regex (?<phone>ddd-ddd-dddd) would match telephone numbers, such as 123-123-1234, and place the parsed value into the phone field.

Let's look at some examples to understand grok better:

#grok1.conf

input { 
   file{
      path => "D:eslogsmsg.log"
      start_position => "beginning"
      sincedb_path => "NULL"
   }
 
} 

filter {
  grok{
  match => {"message" => "%{TIMESTAMP_ISO8601:eventtime} %{USERNAME:userid}  %{GREEDYDATA:data}" }
  } 
}

output { 
  stdout { 
    codec => rubydebug 
    }
    
} 

If the input line is of the "2017-10-11T21:50:10.000+00:00 tmi_19 001 this is a random message" format, then the output would be as follows:

{
          "path" => "D:\es\logs\msg.log",
    "@timestamp" => 2017-11-24T12:30:54.039Z,
          "data" => "this is a random message
",
      "@version" => "1",
          "host" => "SHMN-IN",
     "messageId" => 1,
     "eventtime" => "2017-10-11T21:50:10.000+00:00",
       "message" => "2017-10-11T21:50:10.000+00:00 tmi_19 001 this is a random message
",
        "userid" => "tmi_19"
}

There is a tool hosted at http://grokdebug.herokuapp.com which helps build grok patterns that match the log.