LogMonitor Integration Service

This is the main zProperty for Advanced Log Searches. When you edit this property, a special edit dialog will pop up that allows you to add and edit advanced log searches.

The following steps must be followed to enter and save the log searches:

If a log file in a search is larger than the specified size (in megabytes), the search will not be performed and a warning event will be generated

In Seconds. If a log search takes longer than the specified number of seconds to return with data, the data is still stored, but a warning event will be generated informing of how long the search took. Default is set to a low number, as most searches should come back quickly, but a good rule of thumb is to set it to slightly shorter than the cycle time to guarantee that searches aren't stepping on each other.

Linux string searches are performed using grep with the switches -P (use Perl regular expression syntax) and -c (suppress normal output; instead print a count of matching lines).

The command used by Linux to do the search is this:

grep -cP SEARCH_PATTERN LOGFILE_NAME

Anything that you enter into the "search pattern" field in the Advanced Log Searches dialog must be able to be substituted for SEARCH_PATTERN in the command above. So, for example, if you are only searching for one word, the following would work:

Search pattern: Error

Test command: grep -cP Error some_logfile.log

Or a regular expression without spaces:

Search pattern: [0-9]+$ (a pattern that will search for a line that has a number at the end of the line)

Test command: grep -cP [0-9]+$ some_logfile.log

But if you are searching for a string with spaces, you would need to enclose it in quotes:

Search pattern: "Error detected"

Test command: grep -cP "Error detected" some_logfile.log

Or if you are searching for a string with double-quotes, you could enclose it in single quotes:

Search pattern: 'Error code "12345" has been found'

Test command: grep -cP 'Error code "12345" has been found' some_logfile.log

Keep in mind that everything in the search pattern will be evaluated as a regular expression, so if you are searching for any characters that are used in regular expressions, they will need to be escaped using a backslash.

Here are some examples:

AIX string searches are performed using egrep with the switch -c (suppress normal output; instead print a count of matching lines).

The command used by AIX to do the search is this:

egrep -c SEARCH_PATTERN LOGFILE_NAME

Anything that you enter into the "search pattern" field in the Advanced Log Searches dialog must be able to be substituted for SEARCH_PATTERN in the command above. So, for example, if you are only searching for one word, the following would work:

Search pattern: Error

Test command: egrep -c Error some_logfile.log

Or a regular expression without spaces:

Search pattern: [0-9]+$ (a pattern that will search for a line that has a number at the end of the line)

Test command: egrep -c [0-9]+$ some_logfile.log

But if you are searching for a string with spaces, you would need to enclose it in quotes:

Search pattern: "Error detected"

Test command: egrep -c "Error detected" some_logfile.log

Or if you are searching for a string with double-quotes, you could enclose it in single quotes:

Search pattern: 'Error code "12345" has been found'

Test command: egrep -c 'Error code "12345" has been found' some_logfile.log

Keep in mind that everything in the search pattern will be evaluated as a regular expression, so if you are searching for any characters that are used in regular expressions, they will need to be escaped using a backslash.

Here are some examples:

Windows string searches are performed by using System.Text.RegularExpressions.Regex(SEARCH_PATTERN), comparing it to the log file, line by line from the last position read, and generating a count of matching lines.

The following is a stripped down version of the PowerShell commands used to generate a string count, and can be used to test your search patterns, replacing LOG_FILE with the log file to search and SEARCH_PATTERN with the pattern you are testing:

$logfile = 'LOG_FILE';
$pattern = 'SEARCH_PATTERN';
$stream = New-Object System.IO.FileStream -ArgumentList $logfile, 'Open', 'Read', 'ReadWrite' -ErrorAction Stop;
$reader = New-Object System.IO.StreamReader -ArgumentList $stream, $true;
$reader.BaseStream.Seek(0, 'Begin') | Out-Null;
$reader.ReadLine() | Out-Null;
$reader.DiscardBufferedData();
$regex = New-Object System.Text.RegularExpressions.Regex($pattern);
$search_count = 0;
while($null -ne ($buffer = $reader.ReadLine())) { if($regex.IsMatch($buffer)) { $search_count++; } }
$reader.Close();
$stream.Close();
$search_count

Anything that you enter into the "search pattern" field in the Advanced Log Searches dialog must be able to be substituted for SEARCH_PATTERN in the command above. So, unlike in Linux, quotes are NOT required in your search pattern, even if there are spaces or special characters in the pattern. To escape a single quote, double it (''). No escaping is required for double quotes.

Keep in mind that everything in the search pattern will be evaluated as a regular expression, so if you are searching for any characters that are used in regular expressions, they will need to be escaped using a backslash.

Here are some examples:

Linux datapoint searches are performed using grep with the switches -P (use Perl regular expression syntax) and -o (print only the matched part of a matching line).

The command used by Linux to do the search is this:

grep -oP SEARCH_PATTERN LOGFILE_NAME

This command will give you ALL matching results, while the LogMonitor ZenPack will only return the last one found, but it will let you know if your search pattern returns what you are expecting.

Anything that you enter into the "search pattern" field in the Advanced Log Searches dialog must be able to be substituted for SEARCH_PATTERN in the command above.

The recommended way to do a datapoint search in Linux is to use lookarounds and the digit character.

(?=sometext) - Lookahead: "sometext" comes immediately after the current position

(?<=sometext) - Lookbehind: "sometext" comes immediately before the current position

(?!sometext) - Negative lookahead: "sometext" definitely does NOT come immediately after the current position

(?<!sometext) - Negative lookbehind: "sometext" definitely does NOT come immediately before the current position

For example, assuming a logfile some_logfile.log with the following entry:

WSVR0605W: Thread "WebContainer: 1" has been active for 612,000 milliseconds and may be hung. There are 3 threads in total in the server that may be hung.

A good search pattern to pull out the number of possible hung threads might be:

'(?<=There are)\d+(?= threads in total in the server that may be hung.)'

And you could test that on the command line as follows:

grep -oP '(?<=There are)\d+(?= threads in total in the server that may be hung.)' some_logfile.log

Which should return:

3

Some things to notice about this pattern:

AIX datapoint searches use a combination of egrep and sed.

The AIX method for datapoint searches has several idiosyncracies:

The commands you can use to test your search pattern in AIX are:

OR

Use #2 only if your search pattern includes double quotes. #1 should be used in all other circumstances.

Also notice that we have EGREP_SEARCH_PATTERN and SED_SEARCH_PATTERN. This is due to the different ways egrep and sed use capturing groups, as mentioned above. Because of these idiosyncracies, we have to take the search pattern that we are entering into the LogMonitor ZenPack and format it differently for each command.

For example, assuming a logfile some_logfile.log with the following entry:

WSVR0605W: Thread "WebContainer: 1" has been active for 612,000 milliseconds and may be hung. There are 3 threads (total) in the server that may be hung.

a good search pattern to pull out the number of possible hung threads might be:

^.*There are (([0-9]*)) threads \(total\) in the server that may be hung.*$

Take note that we have used (()) for our capturing group and backslashes to escape the parentheses in the actual text. So, to test the command, we'll need to format the egrep and sed search patterns as follows

Now we can test our search on the command line as follows:

egrep '^.*There are ([0-9]*) threads \(total\) in the server that may be hung.*$' some_logfile.log | sed -n -e 's/^.*There are \([0-9]*\) threads (total) in the server that may be hung.*$/\1/p' | tail -1

Which should return:

3

Here are some example patterns:

Datapoint searches in Windows also use the System.Text.RegularExpressions object, as in the "string" search above. However, to just test your search pattern, you can use this simpler command, replacing TEST_STRING with a line to test against, and SEARCH_PATTERN with your pattern:

$matches = ''; $found = 'TEST_STRING' -match 'SEARCH_PATTERN'; $matches[0]

Anything that you enter into the "search pattern" field in the Advanced Log Searches dialog must be able to be substituted for SEARCH_PATTERN in the command above.

The recommended way to do a datapoint search in Windows is to use lookarounds and the digit character.

(?=sometext) - Lookahead: "sometext" comes immediately after the current position

(?<=sometext) - Lookbehind: "sometext" comes immediately before the current position

(?!sometext) - Negative lookahead: "sometext" definitely does NOT come immediately after the current position

(?<!sometext) - Negative lookbehind: "sometext" definitely does NOT come immediately before the current position

For example, for a log entry that looks like this:

WSVR0605W: Thread "WebContainer: 1" has been active for 612,000 milliseconds and may be hung. There are 3 threads in total in the server that may be hung.

A good search pattern to pull out the number of possible hung threads might be:

(?<=There are)\d+(?= threads in total in the server that may be hung.)

And you could test that on the command line as follows:

$matches = ''; $found = 'WSVR0605W: Thread "WebContainer: 1" has been active for 612,000 milliseconds and may be hung. There are 3 threads in total in the server that may be hung.' -match '(?<=There are)\d+(?= threads in total in the server that may be hung.)'; $matches[0]

Which should return:3

The significant piece here is the \d+. With System.Text.RegularExpressions, lookahead and lookbehind will NOT be included, and only the digits in between will be returned

For datapoint regular expressions, double-quotes do NOT need to be escaped. Single quotes DO need to be escaped by doubling, and other special characters that are used in regular expressions must be escaped by a backslash.

Here are some example patterns: