Extract Variables

In order to be able to define the action in a dedicated rule, you might first need to extract or modify data from the payload of the event.

This can be achieved by creating a dedicated Extractor in a Rule, where a single or multiple variables are extracted with the help of conditions in the WITH clause. The WITH clause generates variables extracted from the Event based on regular expressions. These variables can then be used to populate an Action payload.

Three simple rules restrict the access and use of the extracted variables:

1. Extracted variables are evaluated after the WHERE clause is parsed. For this reason any extracted variables declared inside the WITH clause are not accessible by the WHERE clause of the same rule

2. The order in which variables are extracted within a WITH clause of a rule is not guaranteed. For this reason, it is not recommended to access a variable extracted in the same WITH clause

3. A rule can use extracted variables declared by previous rules of the ruleset, even in its WHERE clause, provided that:

   • The two rules must belong to the same rule set

   • The rule attempting to use those variables should be executed after the one that declares them

   • The rule that declares the variables must also match the event

Note

All variables declared by a Rule must be resolved, or else the Rule will not be matched.

The syntax for accessing an extracted variable has the form:

_variables.[.<RULE_NAME>].<VARIABLE_NAME>

If the RULE_NAME is omitted, the search of a variable is performed in the current rule.

You may need to extract the desired value from an incoming event in many cases, for example setting a particular status to a host object based on the device battery level reported in the event.

Assuming an SMS Event is received,

{
  "event_type": "sms",
  "created_ms": 1695977917962,
  "payload": {
    "sender": "+393333333333",
    "modem": "GSM1"
    "timestamp": 1695922157429,
    "text": "Device battery level: 56%"
  }
}

an extractor Rule with the regex in the WITH clause is to be created in order to define the battery level value

As result, the output after extracting the variable will be available in the UI:

The extracted variable will serve as a basis for defining the action in another Rule of the same Ruleset, e.g. setting “Critical” status to the host object in case the value is lower than 20%.

For this the extracted variable is to be modified with the help of the Modifiers, e.g. map the extracted value to the status that will be displayed in the NetEye Dashbaord.

For more cases of WITH Modifiers’ usage please consult Post Modifiers.

WITH Clause

There are multiple ways of configuring the regexes to obtain the desired result while extracting a variable, and the following entries are common to all configurations:

• variable name: For an easier traceability use a meaningful name for your variable, expecially if there are multiple variables to extract;

• from: The path to the event data to be processed.

The behavior of an extractor is defined by the following three parameters. Note that all these values are mutually exclusive.

MATCH Extractor

MATCH extractor is used for extracting variables by using an index-based regular expression.

The following fields are available:

• MATCH: the index-based regular expression

• ALL MATCHES: set it to True if you want to return all the occurrences matched by the regex and not only the first one.

• GROUP MATCH IDX: the index of the group matched by the regex that you want to return. It returns all groups if nothing is specified.

With the Event payload being (here and for other examples below):

{
  "text": "Device battery level: 56%"
}

the extractor should be built as below:

with the following extracted variable output

Additionally, with ALL MATCHES set to False

the extracted variable output is:

With ALL MATCHES set to True

the extracted variable output is:

SINGLE KEY MATCH Extractor

This extractor is used to extract the value from a key:value map by searching the key with a regex. The field should contain the regex that must match the key of the value that you want to extract.

Note

The regex can only match exactly one key, otherwise an error will be produced.

With the extractor being built as below:

expected extracted variable output is

NAMED MATCH extractor

This extractor extracts a variable by using a regex with named groups.

The following fields are available:

• NAMED MATCH

• ALL MATCHES: set it to True if you want to return all the occurrences matched by the regex and not only the first one. (See examples of an extractor with ALL MATCHES set to True/False in MATCH extractor).

With the extractor being built as below:

expected extracted variable output is

Post Modifiers

The WITH clause can include a list of modifiers to post-process the extracted value.

Lowercase converts the resulting String to lower case.

ToNumber transforms the resulting String into a number.

Trim removes the whitespace from the start and end of the string.

ReplaceAll replaces all matching substrings with the given text. For this use the following fields:

• Find: the string that you want to replace

• Replace: the string that will replace the Find string

• Regex: if enabled, the Find field will be treated as a regular expression

Map map a string to another string value. It replaces a string with another by looking at a map of Value and Replacement pairs. The default_value is optional. If the value of the variable doesn’t matches a value in the map, then the Default value is applied

With the Event received being

{
  "service_status": "ERROR"
}

you can set the values and replacements in Mapping as follows

to get the following output

DateAndTime converts a timestamp (autodetects if it is in seconds, milliseconds or nanoseconds) to an RFC3339 standard datetime. For example the timestamp 1698933188760, with the Europe/Rome timezone, will become 2023-11-02 14:53:08+01:00 string.