Sometimes, we only want to know that an error was logged. In those cases, grabbing the whole line is good enough. But sometimes, the log line might contain an interesting substring, maybe a number of messages in some queue. A log line might look like this:
2015-12-20 18:15:22 Number of messages in the queue: 445
Theoretically, we could write triggers against the whole line. For example, the following regexp should match when there are 10,000 or more messages:
messages in the queue: [1-9][0-9]{4}
But what if we want to have a different trigger when the message count exceeds 15,000? That trigger would have a regular expression like this:
messages in the queue: (1[5-9]|[2-9].)[0-9]{3}
And if we want to exclude values above 15,000 from our first regular expression, it would become the following:
messages in the queue: 1[0-4][0-9]{3}
That's definitely not easy to maintain. And that's with just two thresholds. But there's an easier way to do this, if all we need is that number. Zabbix log monitoring allows us to extract values by regular expressions. To try this out, let's create a file with some values to extract. Still on A test host, create the /tmp/zabbix_logmon/queue_log file with the following content:
echo "2018-12-1 12:01:13 Number of messages in the queue: 445" >> /tmp/zabbix_logmon/queue_log echo "2018-12-1 12:02:14 Number of messages in the queue: 5445" >> /tmp/zabbix_logmon/queue_log
echo "2018-12-1 12:03:15 Number of messages in the queue: 15445" >> /tmp/zabbix_logmon/queue_log
Now, on to the item, go to Configuration | Hosts, click on Items next to A test host, then click on Create item. Fill in the following:
- Name: Extracting log contents
- Type: Zabbix agent (active)
- Key: log[/tmp/zabbix_logmon/queue_log,"messages in the queue: ([0-9]+)",,,,1]
- Type of information: Log
- Update interval: 1s
We quoted regular expression because it contained square brackets again. The regular expression itself extracts the text messages in the queue, followed by a colon, a space, and a number. The number is included in a capture group; this becomes important in the last parameter. To the key, we added 1 which references the capture group contents. This parameter, output, tells Zabbix not to return the whole line, but only whatever is referenced in that parameter. In this case, that's the number.
When done, click on the Add button at the bottom. Some three minutes later, we could check the history for this item in the latest data page:
Hooray! Extracting the values is working as expected. Writing triggers against them should be much, much easier as well. But one thing to note: for this item, we're unable to see the graphs. The reason is the Type of information property in our log item; we had it set to Log, but that type isn't considered suitable for graphing. Let's change it now.
Go to Configuration | Hosts, click on Items next to A test host, and click on Extracting log contents in the Name column. Change Type of information to Numeric (unsigned), then click on the Update button at the bottom.
Check this item in the latest data section; it should have a Graph link now. But checking that reveals that it has no data. How so? Internally, Zabbix stores values for each type of information separately. Changing that doesn't remove the values, but Zabbix only checks the currently configured type. Make sure to set the correct type of information from the start. To verify that this works as expected, run the following on A test host:
$ echo "2018-12-1 18:16:13 Number of messages in the queue: 113" >> /tmp/zabbix_logmon/queue_log $ echo "2018-12-1 18:17:14 Number of messages in the queue: 213" >> /tmp/zabbix_logmon/queue_log $ echo "2018-12-1 18:18:15 Number of messages in the queue: 150" >> /tmp/zabbix_logmon/queue_log
Checking out this item in the Latest data section, the values should be there and the graph should be available, too. Note that the date and time in our log file entries still doesn't matter; the values will get the current timestamp assigned.
Another way to get data out of a log file could be by making use of the Preprocessing tab in our item as shown in the following screenshot. We have the option here to retrieve a whole log line and then pre-process it with, for example, the Regular expression option to cut out information. Only after our regular expression has run will the information be stored into the database:
