API Reference¶

Overview¶

The Tornado API lets you automate all the tasks that are available for execution over the WEB-GUI by calling the endpoints directly. In this section we will go over all the available endpoints and how to use them to query and modify the Tornado Processing Tree.

All available endpoints can be found by default under http://tornado.neteyelocal:4748/api/v2_beta/

Authorization¶

To perform any action in Tornado you need to provide an authorization token with the correct permissions.The token is a base64 encoded JSON object, which contains the username as well as a map of available authorizations.

Permissions¶

To interact with any parts of the tree you need to be granted a corresponding permission. The following permissions are available in NetEye:

Permission	Description
view	Allows you to retrieve any information about the active Processing Tree
edit	Allows you to modify drafts
test_event_execute_actions	Allows you to execute action when sending test events
admin	Gives you full access to all features of the Tornado api

Authorization Token¶

An Authorization Token can contain multiple authorizations. The Authorization is composed of a set of permission as well as a path for which those permissions apply. The path needs to start with root and gives the path to any node in the Processing Tree. Each following node must then be a new element in the list. The api-user is then prohibited to modify any node outside of that base-path.

The roles property of the authotization is a set of Permissions to which the api-user is restricted.

{
    "user": "<my-tornado-user>",
    "auths": {
        "my-auth": {
            "path": [ "root", "master", "email" ],
            "roles": [ "view", "edit" ]
        }
    }
}

Note

The name in the authorization MUST not be empty. An empty name will result in the 401 HTTP error.

To see how to generate the token, take a look at the Getting Started section.

Best practices

Give every api-user a distinct and recognizible name. This lets you track changes and detect clashes during the editing of the draft.

Give every api-user the minimal permission necessary to perform its task. For example, if a api client only needs to read / export config from one specific tenant, give the client only read permissions for their specific tenant node.

Common Request Types¶

Name	Source	Description
param_auth	Path	The authorization from the header to use.
node_path	Path	A comma separated list of node name that points to a specific node, starting from the root node of the tenant.
rule_name	Path	The name of the Rule you want to access
draft_id	Path	The id of the draft you want to take over for your user

Limitations¶

Tornado does not yet support Multitenancy for the draft editing. Your Tornado installation has exactly one active Processing Tree and zero to one drafts with a dedicated owner of said draft. If another user already owns the draft or another api-client or user takes over the draft, during execution, then future requests will be rejected by the endpoint.

There is no reliable way right now to detect whether a draft contains any changes to the current active Processing Tree. So you might need to take over the existing draft, overwrite it, or ask the user to deploy and delete the current draft before you can continue.

Getting Started¶

This section will help to get started with using the Tornado API to edit the Processing Tree through the whole life cycle of a draft. First we’ll create a draft, add a node, edit that node and then deploy the draft so that our changes will end up in the actual environment.

First of all, we need an authorization token, so that Tornado lets us use the API. That token contains a unique name and its permissions. The name for our client will be api-filter-manager, since it will create a new filter for the master tenant. Since we are only operating on the master node, we restrict the token to that path, so we can’t accidentally edit something else.

TOKEN_CONTENT='{"user":"api-filter-manager","auths":{"master-auth":{"path":["root","master"],"roles":["view","edit"]}}}'
AUTH_TOKEN=$(echo "$TOKEN_CONTENT" | base64 --wrap=0)

Before we can edit the draft, we need to make sure that the draft is not being edited at the moment by another user. To do that we make a request to check that no drafts are currently present:

curl -XGET \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/master-auth"

This request should return an empty array. If not, it’s safest to ask the user to deploy and/or delete the current draft before continuing, so that the execution does not interfere with any ongoing modifications to the draft.

Now that we are sure we have a clean slate to work on, we first need to create our own draft. For this, we make a POST request to the appropriate endpoint:

RESULT=$(curl -XPOST \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/master-auth")
DRAFT_ID=$(echo $RESULT | jq .id --raw-output)

We now have successfully created the draft we want to work on. Next we are going to actually create a new node in the Processing Tree and then edit it. First, we make a POST request to create a new filter node as a child of the master tenant. We just provide the new name for now.

curl -XPOST \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    -H "Content-type: application/json" \
    --data '{"type": "Filter","name":"meaning_of_life","active":true,"description":"","filter":null}' \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/details/master-auth/$DRAFT_ID/master"

Next, let’s add a filter to the filter node, so that the children only receive the event designated to them. We’ll do that by first fetching the data for that node from the API, editing it and then send the edited node to Tornado. The creation could be done in one step, but this helps to better illustrate the interaction between the APIs.

FILTER_NODE=$(curl -XGET \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/details/master-auth/$DRAFT_ID/master,meaning_of_life")

FILTER_NODE=$(echo "$FILTER_NODE" | jq '.filter = {"type":"AND","operators":[{"type":"equals","first":"${event.payload.meaning_of_life}","second":42}]}')

curl -XPUT \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    -H "Content-type: application/json" \
    --data "$FILTER_NODE" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/details/master-auth/$DRAFT_ID/master,meaning_of_life"

Now that we have the node ready for use, we can import some base config we want the filter node to have. We assume this configuration is located in the file filter-base-config.json located in the current working directory.

curl -XPOST \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    -F 'file=@filter-base-config.json' \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/import/master-auth/$DRAFT_ID/master,meaning_of_life"

Now we finally configured our new filter how we wanted. The last step is just to deploy the whole configuration so that it can pick up work in the active production environment. After the deploy, we will remove the old draft so that the next user has a clean slate, when he wants to further edit the Tornado config.

curl -XPOST \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/master-auth/$DRAFT_ID/deploy"

curl -XDELETE \
    -H "Authorization: Bearer $AUTH_TOKEN" \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/master-auth/$DRAFT_ID"

With that we have completed the life cycle of a Tornado config draft. The new filter is now ready to process events for Tornado. For further use, check out all the following Documented API endpoints.

Active Processing Tree¶

The Tornado API provides you an easy way to traverse your Processing Tree.

Get Tree Info¶

This endpoint lets you fetch a small summary about the subtree of the tenant.

Url: GET <base-url>/config/active/tree/info/{param_auth}

Required Permission: view

Parameters: See Common Request Types

Example:

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/active/tree/info/my-auth" \
    -H "Authorization: Bearer <auth-token>"

Result

This endpoint returns a JSON object with the following properties:

Property	Type	Description
filters_count	Integer	The total number of filters in the whole subtree
rules_count	Integer	The total number of rules in the whole subtree

Get Node Details¶

This endpoints lets you fetch all information about a single node in the currently active Processing Tree.

Url: GET <base-url>/config/active/tree/details/{param_auth}/{node_path}

Required Permission: view

Parameters: See Common Request Types

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/active/tree/details/my-auth/root,master" \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns a JSON object of type NodeDetails if a node exists in the given path and the api-user has permission to access it.

Get Child Nodes¶

This endpoints lets you fetch basic information about all the child nodes of a specific filter node.

Url: GET <base-url>/config/active/tree/details/{param_auth}/{node_path}

Required Permission: view

Parameters: See Common Request Types

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/active/tree/details/my-auth/root,master" \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns a JSON list of NodeInfo if the node selected is a filter node and the api-user has permission to access it.

Get Rule¶

This endpoints lets you fetch all information about a single Rule in the currently active Processing Tree.

Url: GET <base-url>/config/active/rule/details/{param_auth}/{ruleset_path}/{rule_name}

Required Permission: view

Parameters: See: Common Request Types

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/active/rule/details/my-auth/root,master,my_ruleset" \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns a Rule from the Processing Tree if the node selected is a Ruleset node, a rule with the given name exists and the api-user has the necessary permissions to access it.

Draft Processing Tree¶

Creating a Draft¶

This endpoint lets you create a new draft for the current active Processing Tree.

Warning

If a draft already exists, this endpoint will overwrite the current draft. Make sure there’s no currently active draft, before creating a new one.

Url: POST <base-url>/config/drafts/{param_auth}

Required Permission: edit

Parameters: See Common Request Types

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/my-auth" \
    -XPOST \
    -H "Authorization: Bearer <auth-token>"

Result

This endpoint returns the draft_id for the created draft.

Property	Type	Description
id	String	The id for the newly created draft.

Retrieving All Available Drafts¶

This endpoint lets you list out all available drafts for a specific authorization.

Url: GET <base-url>/config/drafts/{param_auth}

Required Permission: view

Parameters: See Common Request Types

Example:

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/my-auth" \
    -XGET \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns a list of draft_ids available for the selected authorization.

Taking Over Draft¶

This endpoint lets you take over the ownership of a draft from another user.

Url: POST <base-url>/config/drafts/{param_auth}/{draft_id}/takeover

Required Permission: edit

Parameters: See Common Request Types

Example:

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/my-auth/draft_001/takeover" \
    -XPOST \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns nothing if the takeover succeeded, otherwise it returns an error.

Deploying a Draft¶

This endpoint lets you deploy a draft to be the new active Processing Tree. Note that this endpoint only deploys the draft, and doesn’t delete it automatically

Url: POST <base-url>/config/drafts/{param_auth}/{draft_id}/deploy

Required Permission: edit

Parameters: See Common Request Types

Example:

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/my-auth/draft_001/deploy" \
    -XPOST \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns nothing if the deploy succeeded, otherwise it returns an error.

Deleting a Draft¶

This endpoint allows you to delete an existing draft.

Url: DELETE <base-url>/config/drafts/{param_auth}/{draft_id}

Required Permission: edit

Parameters: See Common Request Types

Example:

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/drafts/my-auth/draft_001" \
    -XDELETE \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns nothing if the deploy succeeded, otherwise it returns an error.

Get Draft Node Details¶

Same as Get Node Details, but for a draft.

Url: GET <base-url>/config/draft/tree/details/{param_auth}/{draft_id}/{node_path}

Required Permission: view

Get Draft Node Children¶

Same as Get Child Nodes, but for a draft.

Url: GET <base-url>/config/draft/tree/children/{param_auth}/{draft_id}/{node_path}

Required Permission: view

Create Node in Draft¶

Create a new child node of an existing node in the draft.

Url: POST <base-url>/config/draft/tree/details/{param_auth}/{draft_id}/{node_path}

Required Permission: edit

Parameters

Type	Source	Description
NodeDetails	Body	Pass the configuration of the node to create

Import Child Node¶

Import an exported configuration as a new child node.

Url: POST <base-url>/config/draft/tree/import/{param_auth}/{draft_id}/{node_path}

Required Permission: edit

Parameters

Name	Source	Description
file	Body	A file to upload

Example

curl -XPOST \
    -H "Authorization: Bearer <auth-token>" \
    -F 'file={"version":"1.0","Ruleset":{"name":"my_ruleset","rules":[]}}' \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/import/my-auth/draft_001/root"

Result: Same as Create Node in Draft

Import Node to Replace¶

This endpoint lets you replace an entire subtree with an exported version of the Tornado Processing Tree.

Url: PUT <base-url>/config/draft/tree/import/{param_auth}/{draft_id}/{node_path}

Required Permission: edit

Parameters: Same as Import Child Node

Edit Node in Draft¶

To edit a node in the draft you can use this endpoint. The endpoint however only lets you edit some details of the node. The operation will neither affect the children of the node, nor the node’s type.

Warning

You always have to supply the full config of the node every time, even for editing just the node name. All the data will be overwritten every time.

Url: PUT <base-url>/config/draft/tree/details/{param_auth}/{draft_id}/{node_path}

Required Permission: edit

Parameters

Type	Source	Description
NodeDetails	Body	Pass the configuration of the node to create

Example

curl -XPOST \
    -H "Authorization: Bearer <auth-token>" \
    --data '{"name":"my_ruleset2"}' \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/details/my-auth/draft_001/root,master,my_ruleset"

Result: This endpoint returns nothing if the edit operation succeeds, otherwise it returns an error.

Get Rule¶

Same as Get Rule, but for a draft.

Url: GET <base-url>/config/draft/rule/details/{param_auth}/{draft_id}/{ruleset_path}/{rule_name}

Required Permission: view

Parameters: See: Common Request Types

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/rule/details/my-auth/draft_001/root,master,my_ruleset/my_rule" \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns a Rule from the draft if the node selected is a Ruleset node, a rule with the given name exists and the api-user has the necessary permissions to access it.

Create Rule¶

Create a new rule in an existing ruleset.

Url: POST <base-url>/config/draft/rule/details/{param_auth}/{draft_id}/{ruleset_path}

Required Permission: edit

Parameters

Type	Source	Description
Rule	Body	Path to the existing ruleset where to create the new rule

Edit Rule¶

Edit an existing rule.

Url: PUT <base-url>/config/draft/rule/details/{param_auth}/{draft_id}/{ruleset_path}/{rule_name}

Required Permission: edit

Parameters

Type	Source	Description
Rule	Body	Path to the existing ruleset where to find the rule to edit

Delete Rule¶

Delete an existing rule.

Url: DELETE <base-url>/config/draft/rule/details/{param_auth}/{draft_id}/{ruleset_path}/{rule_name}

Required Permission: edit

Parameters See: Common Request Types

Example

curl -XDELETE \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/rule/details/my-auth/draft_001/root,master,my_ruleset/my_rule" \
    -H "Authorization: Bearer <auth-token>"

Result: This endpoint returns nothing if the delete operation succeeds, otherwise it returns an error.

Reorder Rule in Ruleset¶

Reorder an existing rule in relation of the other rules of the same ruleset.

Url: PUT <base-url>/config/draft/rule/move/{param_auth}/{draft_id}/{ruleset_path}/{rule_name}

Required Permission: edit

Parameters

Type	Source	Description
ruleset_path	URL	Path to the existing ruleset where to create the new rule
RulePosition	Body	New position of the rule in the ruleset

Example

curl -XPUT \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/rule/move/my-auth/draft_001/root,master,my_ruleset/my_rule" \
    -H "Authorization: Bearer <auth-token>" \
    -H "Content-type: application/json" \
    -d '{"position":"3"}'

Result: This endpoint returns nothing if the edit operation succeeds, otherwise it returns an error.

Export Subtree¶

With this Enpoint you can export a subtree of tornado, starting from a given base node.

Url: PUT <base-url>/config/draft/tree/export/{param_auth}/{draft_id}/{node_path}

Required Permission: view

Parameters: See Common Request Types

Example

curl -XGET \
    -H "Authorization: Bearer <auth-token>" \
    -o export.json \
    "http://tornado.neteyelocal:4748/api/v2_beta/config/draft/tree/export/my-auth/draft_001/root"

Result: Returns a HTTP file attachment containing the config of the entire subtree.

Delete Draft Node¶

Delete a Node in the Draft, along with all of its children.

Url: DELETE <base-url>/config/draft/tree/details/{param_auth}/{draft_id}/{node_path}

Required Permission: edit

Result: This endpoint returns nothing if the request succeeded, otherwise it returns an error.

Test Event API¶

Send Test Event To Active Processing Tree¶

This endpoint lets you process a test event with the current active Tornado Processing Tree.

Url: POST /api/v2_beta/event/active/{param_auth}

Required Permissions: view OR edit

Optional Permissions: test_event_execute_actions to execute the actions if they match

Parameters

Name	Source	Description
param_auth	Path	The authorization from the header to use.
event	Body	A tornado event to process with the Processing Tree
process_type	Body	Either `SkipActions` or `Full`. Whether the actions should be executed. If this is set, then the api user MUST have the `test_event_execute_actions` permission

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/event/active/my-auth" \
-H "Authorization: Bearer <auth-token>" \
-XPOST \
--data '{"event":{"type":"the_event_type","created_ms":123456,"payload":{}},"process_type":"Full"}'

Result

Property	Type	Description
event	The Event Type	The initial event that was sent over the request
result	ProcessedNode	The Processing Tree, starting from the auth root, enriched with data about how the event was processed.

Send Test Event To Draft Processing Tree¶

Process a test event with a draft Tornado Processing Tree.

Url: POST /api/v2_beta/event/draft/{param_auth}/{draft_id}

Required Permissions: view OR edit

Optional Permissions: test_event_execute_actions to execute the actions if they match

Parameters: Same as Send Test Event To Active Processing Tree

Example

curl "http://tornado.neteyelocal:4748/api/v2_beta/event/draft/my-auth/draft_001" \
-H "Authorization: Bearer <auth-token>" \
-XPOST \
--data '{"event":{"type":"the_event_type","created_ms":123456,"payload":{}},"process_type":"Full"}'

Result: Same as Send Test Event To Active Processing Tree

Test Event Types¶

ProcessedNode¶

The content of the processed node vary by node type.

Property	Type	Description
type	String	Either Filter or Ruleset
name	String	The name of the node

ProcessedFilterNode¶

Property	Type	Description
filter.status	String	On of Matched, NotMatched or Ignored
nodes	ProcessedNode[]	A list of the filters child nodes. (Only populated if filter status is Matched)

ProcessedRulesetNode¶

Property	Type	Description
rules.rules	ProcessedRule[]	A list of all the rules in the ruleset with extra processing information.
rules.extracted_vars	Object	A map of all extracted variables

ProcessedRule¶

Property	Type	Description
name	String	The name of the Rule
status	String	On of Matched, NotMatched, PartiallyMatched or NotProcessed
action	Action[]	A list of all actions performed by the rule
message (Optional)	String	An error message if a rule was only partially matched
meta.actions (Optional)	Object[]	A list of resolved rules, with extra metadata for the processing

Troubleshooting¶

Should any error occur, you can consult the Tornado logs to see what went wrong. You can find out how to do this on the page Activate Debug Logging for Tornado.

Tornado API Common Types¶

In this section you can find all types that are either accepted or returned by the Tornado Api.

Rule¶

Property	Type	Description
name	String	The name of the rule
description	String	A short user provided description of the rule
continue	Boolean	Determines whether the rule processing should be terminated if the rule matches
active	Boolean	Determines whether the rule is active at the moment
constraint	[]	A list of all the constraint that determine the actions to be performed or not
actions	[]	A list of all the actions of the rule

Example:

{
    "name": "vmdevents",
    "description": "Log and propagate vmd-alarms.",
    "continue": true,
    "active": true,
    "constraint": {
        "WHERE": {
            "type": "AND",
            "operators": [
                {
                    "type": "equals",
                    "first": "${event.payload.foo}",
                    "second": "bar"
                }
            ]
        },
        "WITH": {
            "variable_1": {
                "from": "${event.payload}",
                "regex": {
                    "type": "Regex",
                    "match": ".*",
                    "group_match_idx": null,
                    "all_matches": false
                },
                "modifiers_post": [
                    {
                        "type": "Lowercase"
                    },
                    {
                        "type": "ReplaceAll",
                        "find": "foo",
                        "replace": "bar",
                        "is_regex": false
                    }
                ]
            }
        }
    },
    "actions": [
        {
            "id": "elasticsearch",
            "payload": {
                "auth": {
                    "jwc": true
                },
                "data": {
                    "id": 2
                },
                "endpoint": "elasticsearch.neteyelocal",
                "index": "foobar"
            }
        }
    ]
}

RuleDetails¶

Property	Type	Description
name	String	The name of the rule
description	String	A short user provided description of the rule
continue	Boolean	Determines whether the rule processing should be terminated if the rule matches
active	Boolean	Determines whether the rule is active at the moment
actions	String[]	A list of the names of all actions the rule will perform

Example:

{
    "name": "vmdevents",
    "description": "Log and propagate vmd-alarms.",
    "continue": true,
    "active": true,
    "actions":[ "archive", "script" ]
}

RulePosition¶

Property	Type	Description
position	Integer	The new position of the rule in the ruleset

Example:

{
    "position": "3"
}

Operator¶

The operators are the same as in the config. You can learn all about them in the Tornado WHERE Conditions section of the userguide

NodeInfo¶

The node info gives a quick summary of a node in the Processing Tree. To distinguish between the two, the NodeDetails Type always contains the property type that can be either Ruleset or Filter respectively. Depending on this property the type can have the following properties:

Property	Type	Description
name	String	The name of the filter
rules_count	Integer	The total number of rules in all the subtree
description (Filter-only)	String	A short user provided description of the filter
children_count (Filter-only)	Integer	The number of direct children of the node
active (Filter-only)	Boolean	Determines whether the filter is active at the moment

Example:

{
    "type":"Filter",
    "name":"api_neteye_cloud",
    "rules_count":1,
    "children_count":1,
    "description":"Used for signaling failures in the api.neteye.cloud endpoint",
    "active":true
}

NodeDetails¶

The node details can either be details for a Ruleset or a Filter. To distinguish between the two, the NodeDetails Type always contains the property type that can be either Ruleset or Filter respectively. Depending on this property the type can have the following properties:

FilterNodeDetails¶

Property	Type	Description
type	String	Always set to `Filter`
name	String	The name of the filter
description	String	A short user provided description of the filter
active	Boolean	Determines whether the filter is active at the moment
filter (Optional)	Operator	The operators the rule matches against

Example:

{
    "type": "Filter",
    "name": "master",
    "description": "Events from tenant: master. [..]",
    "active": true,
    "filter": {
        "type":"equals",
        "first":"${event.metadata.tenant_id}",
        "second":"master"
    }
}

RulesetNodeDetails¶

Property	Type	Description
type	String	Always set to `Ruleset`
name	String	The name of the filter
rules	RuleDetails	A list containing basic information on the rules in the Ruleset.

Example:

{
    "type": "Ruleset",
    "name": "api_neteye_cloud",
    "rules": [
        {
            "name": "foobar",
            "description": "",
            "continue": true,
            "active": true,
            "actions": [
                "elasticsearch"
            ]
        },
        {
            "name": "foo",
            "description": "",
            "continue": true,
            "active": true,
            "actions": [
                "script"
                "archive"
            ]
        }
    ]
}

Action¶

Property	Type	Description
id	String	The action name to execute
payload	Object	The contents of the action

Example:

{
    "id":"archive",
    "payload":{
        "archive_type":"vmd",
        "event":"${event.payload}"
    }
}