User Guide

Development

Processing Tree Configuration

The location of configuration files in the file system is pre-configured in NetEye. NetEye automatically starts Tornado as follows:

  • Reads the configuration from the /neteye/shared/tornado/conf directory

  • Starts the Tornado Engine

  • Searches for Filter and Rule definitions in /neteye/shared/tornado/conf/rules.d

The structure of this last directory reflects the Processing Tree structure. Each subdirectory can contain either:

  • A Filter and a set of sub directories corresponding to the Filter’s children

  • A Ruleset

Each individual Rule or Filter to be included in the processing tree must be saved in its own, JSON formatted file.

Hint

Tornado will ignore all other file types.

For instance, consider this directory structure:

/tornado/config/rules
                 |- node_0
                 |    |- 0001_rule_one.json
                 |    \- 0010_rule_two.json
                 |- node_1
                 |    |- inner_node
                 |    |    \- 0001_rule_one.json
                 |    \- filter_two.json
                 \- filter_one.json

In this example, the processing tree is organized as follows:

  • The root node is a filter named filter_one.

  • The filter filter_one has two child nodes: node_0 and node_1:

    • node_0 is a Ruleset that contains two rules called rule_one and rule_two, with an implicit filter that forwards all incoming events to both of its child rules.

    • node_1 is a filter with a single child named inner_node. Its filter filter_two determines which incoming events are passed to its child node.

  • inner_node is a Ruleset with a single rule called rule_one.

Within a Ruleset, the lexicographic order of the file names determines the execution order. The rule filename is composed of two parts separated by the first _ (underscore) symbol. The first part determines the rule execution order, and the second is the rule name. For example:

  • 0001_rule_one.json -> 0001 determines the execution order, “rule_one” is the rule name

  • 0010_rule_two.json -> 0010 determines the execution order, “rule_two” is the rule name

Note

A Rule name must be unique within its own Ruleset, but two rules with the same name may exist in different Rulesets.

Similar to what happens for Rules, Filter names are also derived from the filenames. However, in this case, the entire filename corresponds to the Filter name.

In the example above, the “filter_one” node is the entry point of the processing tree. When an Event arrives, the Matcher will evaluate whether it matches the filter condition, and will pass the Event to one (or more) of the filter’s children. Otherwise it will ignore it.

A node’s children are processed independently. Thus node_0 and node_1 will be processed in isolation and each of them will be unaware of the existence and outcome of the other. This process logic is applied recursively to every node.

Reading Event Fields

A Rule can access Event fields through the “${” and “}” delimiters. To do so, the following conventions are defined:

  • The ‘.’ (dot) char is used to access inner fields.

  • Keys containing dots are escaped with leading and trailing double quotes.

  • Double quote chars are not accepted inside a key.

For example, given the incoming event:

{
    "type": "trap",
    "created_ms": 1554130814854,
    "payload":{
        "protocol": "UDP",
        "oids": {
            "key.with.dots": "38:10:38:30.98"
        }
    }
}

The rule can access the event’s fields as follows:

  • ${event.type}: Returns trap

  • ${event.payload.protocol}: Returns UDP

  • ${event.payload.oids."key.with.dots"}: Returns 38:10:38:30.98

  • ${event.payload}: Returns the entire payload

  • ${event}: Returns the entire event

  • ${event.metadata.key}: Returns the value of the key key from the metadata. The metadata is a special field of an event created by Tornado to store additional information where needed (e.g. the tenant_id, etc.)

String interpolation

An action payload can also contain text with placeholders that Tornado will replace at runtime. The values to be used for the substitution are extracted from the incoming Events following the conventions mentioned in the previous section; for example, using that Event definition, this string in the action payload:

Received a ${event.type} with protocol ${event.payload.protocol}

produces:

*Received a trap with protocol UDP*

Note

Only values of type String, Number, Boolean and null are valid. Consequently, the interpolation will fail, and the action will not be executed, if the value associated with the placeholder extracted from the Event is an Array, a Map, or undefined.

As already seen in the previous section, the WITH clause generates variables extracted from the Event using regular expressions. There are multiple ways of configuring those regexes to obtain the desired result.

Common entries to all configurations:

  • from: An expression that determines to which value to apply the extractor regex;

  • modifiers_post: A list of String modifiers to post-process the extracted value. See following section for additional details.

In addition, three parameters combined will define the behavior of an extractor:

  • all_matches: whether the regex will loop through all the matches or only the first one will be considered. Accepted values are true and false. If omitted, it defaults to false

  • match, named_match or single_key_match: a string value representing the regex to be executed. In detail:

    • match is used in case of an index-based regex,

    • named_match is used when named groups are present.

    • single_key_match is used to search in a map for a key that matches the regex. In case of a match, the extracted variable will be the value of the map associated with that key that matched the regex. This match will fail if more than one key matches the defined regex.

    Note that all these values are mutually exclusive.

  • group_match_idx: valid only in case of an index-based regex. It is a positive numeric value that indicates which group of the match has to be extracted. If omitted, an array with all groups is returned.

To show how they work and what is the produced output, from now on, we’ll use this hypotetical email body as input:

A critical event has been received:

STATUS: CRITICAL HOSTNAME: MYVALUE2 SERVICENAME: MYVALUE3
STATUS: OK HOSTNAME: MYHOST SERVICENAME: MYVALUE41231

Our objective is to extract from it information about the host status and name, and the service name. We show how using different extractors leads to different results.

Option 1

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": false,
          "match": "STATUS:\\s+(.*)\\s+HOSTNAME:\\s+(.*)SERVICENAME:\\s+(.*)",
          "group_match_idx": 1
        }
      }
  }
}

This extractor:

  • processes only the first match because all_matches is false

  • uses an index-based regex specified by match

  • returns the group of index 1

In this case the output will be the string “CRITICAL”.

Please note that, if the group_match_idx was 0, it would have returned “STATUS: CRITICAL HOSTNAME: MYVALUE2 SERVICENAME: MYVALUE3” as in any regex the group with index 0 always represents the full match.

Option 2

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": false,
          "match": "STATUS:\\s+(.*)\\s+HOSTNAME:\\s+(.*)SERVICENAME:\\s+(.*)"
        }
      }
  }
}

This extractor:

  • processes only the first match because all_matches is false

  • uses an index-based regex specified by match

  • returns an array with all groups of the match because group_match_idx is omitted.

In this case the output will be an array of strings:

[
  "STATUS: CRITICAL HOSTNAME: MYVALUE2 SERVICENAME: MYVALUE3",
  "CRITICAL",
  "MYVALUE2",
  "MYVALUE3"
]

Option 3

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": true,
          "match": "STATUS:\\s+(.*)\\s+HOSTNAME:\\s+(.*)SERVICENAME:\\s+(.*)",
          "group_match_idx": 2
        }
      }
  }
}

This extractor:

  • processes all matches because all_matches is true

  • uses an index-based regex specified by match

  • for each match, returns the group of index 2

In this case the output will be an array of strings:

[
  "MYVALUE2", <-- group of index 2 of the first match
  "MYHOST"    <-- group of index 2 of the second match
]

Option 4

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": true,
          "match": "STATUS:\\s+(.*)\\s+HOSTNAME:\\s+(.*)SERVICENAME:\\s+(.*)"
        }
      }
  }
}

This extractor:

  • processes all matches because all_matches is true

  • uses an index-based regex specified by match

  • for each match, returns an array with all groups of the match because group_match_idx is omitted.

In this case the output will be an array of arrays of strings:

[
  [
    "STATUS: CRITICAL HOSTNAME: MYVALUE2 SERVICENAME: MYVALUE3",
    "CRITICAL",
    "MYVALUE2",
    "MYVALUE3"
  ],
  [
    "STATUS: OK HOSTNAME: MYHOST SERVICENAME: MYVALUE41231",
    "OK",
    "MYHOST",
    "MYVALUE41231"
  ]
]

The inner array, in position 0, contains all the groups of the first match while the one in position 1 contains the groups of the second match.

Option 5

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "named_match": "STATUS:\\s+(?P<STATUS>.*)\\s+HOSTNAME:\\s+(?P<HOSTNAME>.*)SERVICENAME:\\s+(?P<SERVICENAME>.*)"
        }
      }
  }
}

This extractor:

  • processes only the first match because all_matches is omitted

  • uses a regex with named groups specified by named_match

In this case the output is an object where the group names are the property keys:

{
  "STATUS": "CRITICAL",
  "HOSTNAME": "MYVALUE2",
  "SERVICENAME: "MYVALUE3"
}

Option 6

{
  "WITH": {
      "server_info": {
        "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": true,
          "named_match": "STATUS:\\s+(?P<STATUS>.*)\\s+HOSTNAME:\\s+(?P<HOSTNAME>.*)SERVICENAME:\\s+(?P<SERVICENAME>.*)"
        }
      }
  }
 }

This extractor:

  • processes all matches because all_matches is true

  • uses a regex with named groups specified by named_match

In this case the output is an array that contains one object for each match:

[
  {
    "STATUS": "CRITICAL",
    "HOSTNAME": "MYVALUE2",
    "SERVICENAME: "MYVALUE3"
  },
  {
    "STATUS": "OK",
    "HOSTNAME": "MYHOST",
    "SERVICENAME: "MYVALUE41231"
  },
]

The ‘WITH’ Clause - Post Modifiers

The WITH clause can include a list of String modifiers to post-process the extracted value. The available modifiers are:

  • Lowercase: it converts the resulting String to lower case. Syntax:

    {
    "type": "Lowercase"
}

  • Map: it maps a string to another string value. Syntax:

    {
      "type": "Map",
      "mapping": {
        "Critical": "2",
        "Warning": "1",
        "Clear": "0",
        "Major": "2",
        "Minor": "1"
      },
      "default_value": "3"
}

    The default_value is optional; when provided, it is used to map values that do not have a corresponding key in the mapping field. When not provided, the extractor will fail if a specific mapping is not found.

  • ReplaceAll: it returns a new string with all matches of a substring replaced by the new text; the find property is parsed as a regex if is_regex is true, otherwise it is evaluated as a static string. Syntax:

    {
    "type": "ReplaceAll",
    "find": "the string to be found",
    "replace": "to be replaced with",
    "is_regex": false
}

    In addition, when is_regex is true, is possible to interpolate the regex captured groups in the replace string, using the $<position> syntax, for example:

    {
    "type": "ReplaceAll",
    "find": "(?P<lastname>[^,\\s]+),\\s+(?P<firstname>\\S+)",
    "replace": "firstname: $2, lastname: $1",
    "is_regex": true
}

    Valid forms of the replace field are:

    • extract from event: ${events.payload.hostname_ext}

    • use named groups from regex: $digits and other

    • use group positions from regex: $1 and other

  • ToNumber: it transforms the resulting String into a number. Syntax:

    {
    "type": "ToNumber"
}

  • Trim: it trims the resulting String. Syntax:

    {
    "type": "Trim"
}

A full example of a WITH clause using modifiers is:

{
  "WITH": {
      "server_info": {
       "from": "${event.payload.email.body}",
        "regex": {
          "all_matches": false,
          "match": "STATUS:\s+(.*)\s+HOSTNAME:\s+(.*)SERVICENAME:\s+(.*)",
          "group_match_idx": 1
        },
        "modifiers_post": [
            {
              "type": "Lowercase"
            },
            {
              "type": "ReplaceAll",
              "find": "to be found",
              "replace": "to be replaced with",
              "is_regex": false
            },
            {
              "type": "Trim"
            }
        ]
       }
     }
  }

This extractor has three modifiers that will be applied to the extracted value. The modifiers are applied in the order they are declared, so the extracted string will be transformed in lowercase, then some text replaced, and finally, the string will be trimmed.

Complete Rule Example 1

An example of a valid Rule in a JSON file is:

{
  "description": "This matches all emails containing a temperature measurement.",
  "continue": true,
  "active": true,
  "constraint": {
    "WHERE": {
      "type": "AND",
      "operators": [
        {
          "type": "equals",
          "first": "${event.type}",
          "second": "email"
        }
      ]
    },
    "WITH": {
      "temperature": {
        "from": "${event.payload.body}",
        "regex": {
          "match": "[0-9]+\\sDegrees",
          "group_match_idx": 0
        }
      }
    }
  },
  "actions": [
    {
      "id": "Logger",
      "payload": {
        "type": "${event.type}",
        "subject": "${event.payload.subject}",
        "temperature:": "The temperature is: ${_variables.temperature} degrees"
      }
    }
  ]
}

This creates a Rule with the following characteristics:

  • Its unique name is ‘emails_with_temperature’. There cannot be two rules with the same name.

  • An Event matches this Rule if, as specified in the WHERE clause, it has type “email”, and as requested by the WITH clause, it is possible to extract the “temperature” variable from the “event.payload.body” with a non-null value.

  • If an Event meets the previously stated requirements, the matcher produces an Action with id “Logger” and a payload with the three entries type, subject and temperature.

API

Self-Monitoring API

The monitoring endpoints allow you to monitor the health of Tornado. They provide information about the status, activities, logs and metrics of a running Tornado instance. Specifically, they return statistics about latency, traffic, and errors.

Available endpoints:

Ping endpoint

This endpoint returns a simple message “pong - “ followed by the current date in ISO 8601 format.

Details:

  • name : ping

  • path : /monitoring/ping

  • response type: JSON

  • response example:

    {
  "message": "pong - 2019-04-12T10:11:31.300075398+02:00",
}

Metrics endpoint

This endpoint returns tornado metrics in the Prometheus text format Details:

  • name : metrics/prometheus

  • path : /monitoring/v1/metrics/prometheus

  • response type: Prometheus text format

  • response example:

    # HELP events_processed_counter Events processed count
# TYPE events_processed_counter counter
events_processed_counter{app="tornado",event_type="icinga_process-check-result"} 1
# HELP events_processed_duration_seconds Events processed duration
# TYPE events_processed_duration_seconds histogram
events_processed_duration_seconds_bucket{app="tornado",event_type="icinga_process-check-result",le="0.5"} 1
events_processed_duration_seconds_bucket{app="tornado",event_type="icinga_process-check-result",le="0.9"} 1
events_processed_duration_seconds_bucket{app="tornado",event_type="icinga_process-check-result",le="0.99"} 1
events_processed_duration_seconds_bucket{app="tornado",event_type="icinga_process-check-result",le="+Inf"} 1
events_processed_duration_seconds_sum{app="tornado",event_type="icinga_process-check-result"} 0.000696327
events_processed_duration_seconds_count{app="tornado",event_type="icinga_process-check-result"} 1
# HELP events_received_counter Events received count
# TYPE events_received_counter counter
events_received_counter{app="tornado",event_type="icinga_process-check-result",source="http"} 1
# HELP http_requests_counter HTTP requests count
# TYPE http_requests_counter counter
http_requests_counter{app="tornado"} 1
# HELP http_requests_duration_secs HTTP requests duration
# TYPE http_requests_duration_secs histogram
http_requests_duration_secs_bucket{app="tornado",le="0.5"} 1
http_requests_duration_secs_bucket{app="tornado",le="0.9"} 1
http_requests_duration_secs_bucket{app="tornado",le="0.99"} 1
http_requests_duration_secs_bucket{app="tornado",le="+Inf"} 1
http_requests_duration_secs_sum{app="tornado"} 0.001695673
http_requests_duration_secs_count{app="tornado"} 1

The following metrics are provided:

  • events_received_counter : total number of received events grouped by source (nats, tcp, http) and event type

  • events_processed_counter : total number of processed events grouped by event type

  • events_processed_duration_seconds_sum : total time spent for event processing

  • events_processed_duration_seconds_count : total number of processed events

  • invalid_events_received_counter : total number of received event with a not valid format. This can be caused, for example, by a not valid JSON representation or by the missing of mandatory fields

  • actions_received_counter : total number of received actions grouped by type

  • actions_processed_counter : total number of processed actions grouped by type and outcome (failure or success)

  • actions_processing_attempts_counter : total number of attempts to execute an action grouped by type and outcome (failure or success). This number can be greater than the total number of received actions because the execution of a single action can be attempted multiple times based on the defined Retry Strategy

Many other metrics can be derived from these ones, for example:

  • events_processed_counter - events_received_counter = events waiting to be processed

  • events_processed_duration_seconds_sum / events_processed_duration_seconds_count = mean processing time for an event

Tornado Backend API v1

The Tornado Backend contains endpoints that allow you to interact with Tornado through REST endpoints.

In this section we describe the version 1 of the Tornado Backend APIs.

Tornado ‘Auth’ Backend API

The ‘auth’ APIs require the caller to pass an authorization token in the headers in the format:

Authorization : Bearer TOKEN_HERE

The token should be a base64 encoded JSON with this user data:

{
  "user": "THE_USER_IDENTIFIER",
  "roles": ["ROLE_1", "ROLE_2", "ROLE_2"]
}

In the coming releases the current token format will be replaced by a JSON Web Token (JWT).

Tornado ‘Config’ Backend API

The ‘config’ APIs require the caller to pass an authorization token in the headers as in the ‘auth’ API.

Working with configuration and drafts

These endpoints allow working with the configuration and the drafts

Endpoint: get the current Tornado configuration

  • HTTP Method: GET

  • path : /api/v1_beta/config/current

  • response type: JSON

  • response example:

    {
  "type": "Rules",
  "rules": [
    {
      "name": "all_emails",
      "description": "This matches all emails",
      "continue": true,
      "active": true,
      "constraint": {
        "WHERE": {
          "type": "AND",
          "operators": [
            {
              "type": "equal",
              "first": "${event.type}",
              "second": "email"
            }
          ]
        },
        "WITH": {}
      },
      "actions": [
        {
          "id": "Logger",
          "payload": {
            "subject": "${event.payload.subject}",
            "type": "${event.type}"
          }
        }
      ]
    }
  ]
}

Endpoint: get list of draft ids

  • HTTP Method: GET

  • path : /api/v1_beta/config/drafts

  • response type: JSON

  • response: An array of String ids

  • response example:

    ["id1", "id2"]

Endpoint: get a draft by id

  • HTTP Method: GET

  • path : /api/v1_beta/config/drafts/{draft_id}

  • response type: JSON

  • response: the draft content

  • response example:

    {
  "type": "Rules",
  "rules": [
    {
      "name": "all_emails",
      "description": "This matches all emails",
      "continue": true,
      "active": true,
      "constraint": {
        "WHERE": {},
        "WITH": {}
      },
      "actions": []
    }
  ]
}

Endpoint: create a new draft and return the draft id. The new draft is an exact copy of the current configuration; anyway, a root Filter node is added if not present.

  • HTTP Method: POST

  • path : /api/v1_beta/config/drafts

  • response type: JSON

  • response: the draft content

  • response example:

    {
  "id": "id3"
}

Endpoint: update an existing draft

  • HTTP Method: PUT

  • path : /api/v1_beta/config/drafts/{draft_id}

  • request body type: JSON

  • request body: The draft content in the same JSON format returned by the GET /api/v1_beta/config/drafts/{draft_id} endpoint

  • response type: JSON

  • response: an empty json object

Endpoint: delete an existing draft

  • HTTP Method: DELETE

  • path : /api/v1_beta/config/drafts/{draft_id}

  • response type: JSON

  • response: an empty json object

Endpoint: take over an existing draft

  • HTTP Method: POST

  • path : /api/v1_beta/config/drafts/{draft_id}/take_over

  • response type: JSON

  • response: an empty json object

Endpoint: deploy an existing draft

  • HTTP Method: POST

  • path : /api/v1_beta/config/drafts/{draft_id}/deploy

  • response type: JSON

  • response: an empty json object

Tornado ‘Event’ Backend API

Send Test Event Endpoint

Endpoint: match an event on the current Tornado Engine configuration

  • HTTP Method: POST

  • path : /api/v1_beta/event/current/send

  • request type: JSON

  • request example:

    {
    "event": {
      "type": "the_event_type",
      "created_ms": 123456,
      "payload": {
        "value_one": "something",
        "value_two": "something_else"
      }
    },
    "process_type": "SkipActions"
}

    Where the event has the following structure:

    • type: The Event type identifier

    • created_ms: The Event creation timestamp in milliseconds since January 1, 1970 UTC

    • payload: A Map<String, Value> with event-specific data

    • process_type: Can be Full or SkipActions:

      • Full: The event is processed and linked actions are executed

      • SkipActions: The event is processed but actions are not executed

  • response type: JSON

  • response example:

    {
 "event": {
   "type": "the_event_type",
   "created_ms": 123456,
   "payload": {
     "value_one": "something",
     "value_two": "something_else"
   }
 },
 "result": {
   "type": "Rules",
   "rules": {
     "rules": {
       "emails_with_temperature": {
         "rule_name": "emails",
         "status": "NotMatched",
         "actions": [],
         "message": null
       },
       "archive_all": {
         "rule_name": "archive_all",
         "status": "Matched",
         "actions": [
           {
             "id": "archive",
             "payload": {
               "archive_type": "one",
               "event": {
                 "created_ms": 123456,
                 "payload": {
                   "value_one": "something",
                   "value_two": "something_else"
                 },
                 "type": "the_event_type"
               }
             }
           }
         ],
         "message": null
       }
     },
     "extracted_vars": {}
   }
 }
}

Endpoint: match an event on a specific Tornado draft

  • HTTP Method: POST

  • path : /api/v1_beta/event/drafts/{draft_id}/send

  • request type: JSON

  • request/response example: same request and response of the /api/v1_beta/event/current/send endpoint

Tornado ‘RuntimeConfig’ Backend API

These endpoints allow inspecting and changing the tornado configuration at runtime. Please note that whatever configuration change performed with these endpoints will be lost when tornado is restarted.

Get the logger configuration

Endpoint: get the current logger level configuration

  • HTTP Method: GET

  • path: /api/v1_beta/runtime_config/logger

  • response type: JSON

  • response example:

    {
  "level": "info",
  "stdout_enabled": true,
  "apm_enabled": false
}

Set the logger level

Endpoint: set the current logger level configuration

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/logger/level

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

    {
  "level": "warn,
  tornado=trace"
}

Set the logger stdout output

Endpoint: Enable or disable the logger stdout output

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/logger/stdout

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

    {
  "enabled": true
}

Set the logger output to Elastic APM

Endpoint: Enable or disable the logger output to Elastic APM

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/logger/apm

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

    {
  "enabled": true
}

Set the logger configuration with priority to Elastic APM

Endpoint: This will disable the stdout and enable the Elastic APM logger; in addition, the logger level will be set to the one provided, or to “info,tornado=debug” if not present.

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/logger/set_apm_priority_configuration

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

{
  "logger_level": true
}

Set the logger configuration with priority to stdout

Endpoint: this will disable the Elastic APM logger and enable the stdout; in addition, the logger level will be set to the one provided in the configuration file.

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/logger/set_stdout_priority_configuration

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

    {}

Set the Smart Monitoring Executor status

Endpoint: this activates and deactivates the Smart Monitoring Executor. When the Executor is in active state, it will execute incoming Actions as soon as they are produced. When instead the Executor is in inactive state, all incoming Actions will be queued until the Executor returns in active state.

  • HTTP Method: POST

  • path: /api/v1_beta/runtime_config/executor/smart_monitoring

  • response: http status code 200 if the request was performed correctly

  • request body type: JSON

  • request body:

    {
   "active": false
}

Tornado Backend APIv2

The version 2 of the APIs allows for a granular querying of the processing tree, reducing the end-to-end latency with respect to the version 1 of the APIs.

The Tornado Backend APIs v2 furthermore allow for granular authorization on the processing tree for different users.

Tornado ‘Auth’ Backend APIv2

The version 2 of the Tornado APIs require the caller to pass an authorization token in the headers in the format:

Authorization : Bearer TOKEN_HERE

The token should be a base64 encoded JSON with the following user data:

{
  "user": "THE_USER_IDENTIFIER",
  "auths": {
    "PARAM_AUTH_1": {
      "path": [
        "root"
      ],
      "roles": [
        "view",
        "edit",
        "test_event_execute_actions"
      ]
    },
    "PARAM_AUTH_2": {
      "path": [
        "root",
        "filter1",
        "filter2"
      ],
      "roles": [
        "view",
        "test_event_execute_actions"
      ]
    }
  },
  "preferences": {
    "language": "en_US"
  }
}

Tornado ‘Config’ Backend APIv2

The ‘config’ APIs require the caller to pass an authorization token in the headers as in the ‘auth’ API.

Reading the current configuration

These endpoints allow to read the current configuration tree

Endpoint: get the current configuration tree of the root node.

  • HTTP Method: GET

  • path : /api/v2_beta/config/active/tree/children/{param_auth}

    • where {param_auth} is the key of one of the auth fields contained in the authentication header

  • response type: JSON

  • request permissions:

    • To call this endpoint you need the ConfigView permission.

  • response example:

    [
  {
    "type": "Filter",
    "name": "root",
    "rules_count": 60,
    "description": "This is the root node",
    "children_count": 2
  }
]

Endpoint: get the current configuration tree of a specific node. Node names must be separated by a comma.

  • HTTP Method: GET

  • path : /api/v2_beta/config/active/tree/children/{param_auth}/root,foo

    • where {param_auth} is the key of one of the auth fields contained in the authentication header

  • response type: JSON

  • request permissions:

    • To call this endpoint you need the ConfigView permission.

  • response example:

    [
  {
    "type": "Filter",
    "name": "foo",
    "rules_count": 40,
    "description": "This is the foo node",
    "children_count": 4
  }
]

Tornado ‘Rule Details’ Backend APIv2

Reading a rule details

Endpoint: get single rule details giving the rule name and the ruleset path related to the current configuration tree. The ‘rule details’ APIs require the caller to pass an authorization token in the headers as in the ‘auth’ API.

  • HTTP Method: GET

  • path: /api/v2_beta/config/active/rule/details/admins/root,foo,rulesetA/foobar_rule

  • response type: JSON

  • response example:

    {
    "name":"foobar_rule",
    "description":"foobar_rule description",
    "continue":true,
    "active":true,
    "constraint":{
        "type": "AND",
        "operators": [
            {
              "type": "equal",
              "first": "${event.type}",
              "second": "email"
            }
        ],
        "WITH":{}
    },
    "actions":[
        {
            "id": "Logger",
            "payload": {
                "subject": "${event.payload.subject}",
                "type": "${event.type}"
            }
        }
    ]
}

Tornado ‘Node Details’ Backend APIv2

Reading the current configuration details

Endpoint: The ‘node details’ APIs require the caller to pass an authorization token in the headers as in the ‘auth’ API.

  • HTTP Method: GET

  • path : /api/v2_beta/config/active/tree/details/{param_auth}/root,foo

    • where {param_auth} is the key of one of the auth fields contained in the authentication header

  • response type: JSON

  • request permissions:

    • To call this endpoint you need the ConfigView permission.

  • response example:

     {
    "type":"Filter",
    "name":"foo",
    "description":"This filter allows events for Linux hosts",
    "active":true,
    "filter":{
        "type":"equals",
        "first":"${event.metadata.os}",
        "second":"linux"
    }
}

Tornado ‘Event’ Backend APIv2

Send Test Event Endpoint V2

Endpoint: match an event on the current Tornado Engine configuration

  • HTTP Method: POST

  • path : /api/v2_beta/event/active/{param_auth}

    • where {param_auth} is the key of one of the auth fields contained in the authentication header

  • request type: JSON

  • request permissions:

    • To call this endpoint with a process_type equal to SkipActions you need at least one of the following permissions:

      • ConfigView

      • ConfigEdit

    • To call this endpoint with a process_type equal to Full you need the TestEventExecuteActions permission in addition to the permissions needed for the SkipActions process_type.

  • request example:

    {
    "event": {
      "type": "the_event_type",
      "created_ms": 123456,
      "payload": {
        "value_one": "something",
        "value_two": "something_else"
      }
    },
    "process_type": "SkipActions"
}

    Where the event has the following structure:

    • type: The Event type identifier

    • created_ms: The Event creation timestamp in milliseconds since January 1, 1970 UTC

    • payload: A Map<String, Value> with event-specific data

    • process_type: Can be Full or SkipActions:

      • Full: The event is processed and linked actions are executed

      • SkipActions: The event is processed but actions are not executed

  • response type: JSON

  • response example:

    {
 "event": {
   "type": "the_event_type",
   "created_ms": 123456,
   "payload": {
     "value_one": "something",
     "value_two": "something_else"
   }
 },
 "result": {
   "type": "Rules",
   "rules": {
     "rules": {
       "emails_with_temperature": {
         "rule_name": "emails",
         "status": "NotMatched",
         "actions": [],
         "message": null
       },
       "archive_all": {
         "rule_name": "archive_all",
         "status": "Matched",
         "actions": [
           {
             "id": "archive",
             "payload": {
               "archive_type": "one",
               "event": {
                 "created_ms": 123456,
                 "payload": {
                   "value_one": "something",
                   "value_two": "something_else"
                 },
                 "type": "the_event_type"
               }
             }
           }
         ],
         "message": null
       }
     },
     "extracted_vars": {}
   }
 }
}

Endpoint: match an event on a specific Tornado draft

  • HTTP Method: POST

  • path : /api/v2_beta/event/drafts/{draft_id}/{param_auth}

  • request type: JSON

  • request permissions:

    • To call this endpoint with a process_type equal to SkipActions you need to be the owner of the draft and have the ConfigEdit permission.

    • To call this endpoint with a process_type equal to Full you need the TestEventExecuteActions permission in addition to the permissions needed for the SkipActions process_type.

  • request/response example: same request and response of the /api/v2_beta/event/active/{param_auth} endpoint

Tornado ‘Tree Info’ Backend APIv2

Endpoint: The ‘tree info’ APIs require the caller to pass an authorization token in the headers as in the ‘auth’ API.

  • HTTP Method: GET

  • path : /api/v2_beta/config/active/tree/info/{param_auth}

    • where {param_auth} is the key of one of the auth fields contained in the authentication header

  • response type: JSON

  • request permissions:

    • To call this endpoint you need the ConfigView permission.

  • response example:

    {
   "rules_count": 14,
   "filters_count": 2
}