User Guide

Advanced Topics

Elasticsearch security helper tool

The secure communication provided by the X-Pack Security requires additional parameters such as authentication certificates to interact with the Elastic Stack APIs. We have developed a few helper tools, based on curl, to simplify your interaction with the APIs.

The Elasticsearch helper script lets you omit all the authentication parameters for the admin user, which would otherwise be required.

Location: /usr/share/neteye/elasticsearch/scripts/es_curl.sh

The NetEye helper script can be used instead if you only need read permission for the fields @timestamp and host on the Logstash index entries. This script is used by NetEye for self-monitoring activities.

Location: /usr/share/neteye/elasticsearch/scripts/es_neteye_curl.sh

Elasticsearch System indices on Single Nodes

Elasticsearch System indices are a type of indices meant to be used only internally by the components of the Elastic Stack. They may be associated with some System template which specifies their settings.

Some System templates may require System indices to have at least one replica, which in simple terms means that the index must be replicated on more than one node (note that this may apply also for non-System templates and indices). While having replicas is straightforward on a NetEye Cluster, on Single Node installations this is not possible, which causes these indices to go in yellow state due to the so-called unassigned shards warning.

Since on Single Node installations this warning cannot be solved by assigning shards to other nodes, the solution to the problem is to tell Elasticsearch that non-replicated indices should be allowed.

NetEye applies this solution by default during the neteye_secure_install on the templates and indices which have this problem. However, some templates and indices are created only when some features of the Elastic Stack are triggered by the user and must be manually fixed by the Elastic Stack administrator.

To facilitate the job of admin though, NetEye provides a script that given the name of an Elasticsearch template, modifies the settings of the template itself and of all the matching indices to allow zero replicas on Single Node installations.

This goal is achieved by the script by setting the option index.auto_expand_replicas to 0-1.

Suppose you identified that the Elasticsearch template that causes the index to have the number of replicas set to one (or more) is named .siem-signals-default. You can then call the script as follows:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh ".siem-signals-default"

Note

The script works only with the composable index templates introduced in Elasticsearch 7.8 and does not support the Legacy index templates.

Moreover, the script supports the update of multiple Index Templates at once. To perform such operation simply pass the multiple Index Templates names are arguments, like this:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh <index_template_name_1> <index_template_name_2> <index_template_name_3>

Kibana Keystore Usage

The Kibana Keystore feature comes with a keybana-keystore tool, which permits to manage the settings in the keystore.

If your installation is a NetEye Cluster, you are advised to use kibana-keystore tool only from the cluster nodes where the Kibana resource is active.

Using the keybana-keystore tool from nodes where Kibana is not running will have no effect on the Kibana Keystore configuration.

El Proxy Security

The El Proxy module is automatically configured upon installation to use certificates for improved security and to comply with standard NetEye policies. This section describes some details of the configuration and can be used to modify the setup if required.

TLS configuration

The El Proxy server can start in HTTP or HTTPS mode; this is configured in the config web_server.tls section.

The available modes are:

  • None: The El Proxy server starts with TLS disabled. Example:

    [web_server.tls]
    type = "None"
    
  • PemCertificatePath: The El Proxy server starts with TLS enabled using the PEM certificates read from the local file system. When this method is used, the following information must be provided:

    • certificate_path: path to the server public certificate

    • private_key_path: path to the server private key

    Example:

    [web_server.tls]
    type = "PemCertificatePath"
    certificate_path = "/path/to/certs/ebp_server.crt.pem"
    private_key_path = "/path/to/certs/private/ebp_server.key.pem"
    

Authentication to Elasticsearch

When the Elasticsearch client is created, the authentication method to be used to connect to Elasticsearch needs to be specified. The authentication method defined in the configuration file is only used for the serve command.

The available authentication methods are:

  • None: the client connects to Elasticsearch without authentication. Example:

    [elasticsearch.auth]
    type = "None"
    
  • BasicAuth: the client authenticates to Elasticsearch with username and password. When this method is used, the following information must be provided:

    • username: name of the Elasticsearch user

    • password: the password for the Elasticsearch user

    [elasticsearch.auth]
    type = "BasicAuth",
    username = "myuser",
    password = "mypassword"
    
  • PemCertificatePath: the client connects to Elasticsearch using the PEM certificates read from the local file system. When this method is used, the following information must be provided:

    • certificate_path: path to the public certificate accepted by Elasticsearch

    • private_key_path: path to the corresponding private key

    Example:

    [elasticsearch.auth]
    type = "PemCertificatePath",
    certificate_path = "/path/to/certs/ebp.crt.pem",
    private_key_path = "/path/to/certs/private/ebp.key.pem",
    

El Proxy Verification

In El Proxy, the verify command can be used to understand if the underlying blockchain has been corrupted. In the remainder, some advanced aspects of the verification are outlined.

First Iteration Retrieval

During the verification process, one of the first steps performed by El Proxy is the retrieval of the first expected iteration. This aspect is particularly important when considering the retention policies, since the first expected iteration needs to be calculated based on the logs that we still expect to find in Elasticsearch. The following diagram outlines the first steps of the verification process which involve the retrieval of the first iteration and the various errors or warnings raised.

../_images/elproxy_blockchain_verification_first_iteration.png

Fig. 209 El Proxy verification process - First iteration retrieval.

(*) for more information please refer to the following Must Exists vs May Exist section

Warnings and Error Codes

The verification process may complete successfully, emit an error or a warning. Errors thrown by El Proxy lead to a failed verification, while warnings may appear also on successful completion.

The following table reports the warnings that could be reported by El Proxy during the verification process, along with the possible causes and actions that can be taken to address the issue.

Table 78 El Proxy Verification Warnings

Warning Code

Description

Actions

UntrustedFirstIterationNoStateHistory

The blockchain state history file was not found and therefore the first iteration was taken from Elasticsearch, leading to an untrusted first iteration.

After a successful verification, if the blockchain state history file was never written, no further actions are required. Otherwise, a manual investigation of the causes that brought to the absence of the blockchain state history file is required.

UntrustedFirstIterationInsufficientInfo

The blockchain state history file was found, but it does not contain enough information to assess the expected first iteration of the blockchain, which was thus retrieved from Elasticsearch and therefore it is untrusted.

This warning appears when the time between two consecutive verifications exceeds the retention period, leading to an outdated history and therefore to the retrieval from Elasticsearch. Increasing the frequency of the verifications will avoid the presence of the warning.

UnknownFirstIteration

El Proxy was unable to identify the first expected iteration, neither by fetching the Blockchain State History, nor by retrieving logs from Elasticsearch.

This warning appears if the blockchain is new and no logs have been indexed, a case in which no actions are required. As an alternative, if all logs have been deleted and this is the first verification, the history was not yet written, a case in which a manual investigation on the causes is necessary.

EmptyBlockchain

The first expected iteration and all the subsequent iterations were not found in the blockchain, thus the first expected iteration had been computed based on the last iteration present before the retention period of the blockchain.

This warning appears if no logs have been written within the retention period, requiring no further actions. In case you expected logs to be present, you should perform manual investigation to assess if all logs within the retention period might have been deleted.

Similarly, the following table reports the errors that could be reported by El Proxy during the verification process, along with the possible causes and actions that can be taken to address the issue.

Table 79 El Proxy Verification Errors

Error Code

Description

Actions

EmptyBlockchainCorruption

The blockchain is empty and the first expected iteration was not found, leading to a failed verification

There are mainly two causes that could lead to this error and in both cases a manual investigation of what has happened is required: the blockchain state history file contains wrong information, leading to a wrong expected first iteration, or all logs in the blockchain have been deleted

VerificationFailedError

A corruption was found in the blockchain and therefore the verification process failed

A manual investigation of the missing logs is required