Advanced Topics¶

Elasticsearch security helper tool¶

The secure communication provided by the X-Pack Security requires additional parameters such as authentication certificates to interact with the Elastic Stack APIs. We have developed a few helper tools, based on curl, to simplify your interaction with the APIs.

The Elasticsearch helper script lets you omit all the authentication parameters for the admin user, which would otherwise be required.

Location: /usr/share/neteye/elasticsearch/scripts/es_curl.sh

The NetEye helper script can be used instead if you only need read permission for the fields @timestamp and host on the Logstash index entries. This script is used by NetEye for self-monitoring activities.

Location: /usr/share/neteye/elasticsearch/scripts/es_neteye_curl.sh

Elasticsearch System indices on Single Nodes¶

Elasticsearch System indices are a type of indices meant to be used only internally by the components of the Elastic Stack. They may be associated with some System template which specifies their settings.

Some System templates may require System indices to have at least one replica, which in simple terms means that the index must be replicated on more than one node (note that this may apply also for non-System templates and indices). While having replicas is straightforward on a NetEye Cluster, on Single Node installations this is not possible, which causes these indices to go in yellow state due to the so-called unassigned shards warning.

Since on Single Node installations this warning cannot be solved by assigning shards to other nodes, the solution to the problem is to tell Elasticsearch that non-replicated indices should be allowed.

NetEye applies this solution by default during the neteye_secure_install on the templates and indices which have this problem. However, some templates and indices are created only when some features of the Elastic Stack are triggered by the user and must be manually fixed by the Elastic Stack administrator.

To facilitate the job of admin though, NetEye provides a script that given the name of an Elasticsearch template, modifies the settings of the template itself and of all the matching indices to allow zero replicas on Single Node installations.

This goal is achieved by the script by setting the option index.auto_expand_replicas to 0-1.

Suppose you identified that the Elasticsearch template that causes the index to have the number of replicas set to one (or more) is named .siem-signals-default. You can then call the script as follows:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh ".siem-signals-default"

Note

The script works only with the composable index templates introduced in Elasticsearch 7.8 and does not support the Legacy index templates.

Moreover, the script supports the update of multiple Index Templates at once. To perform such operation simply pass the multiple Index Templates names are arguments, like this:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh <index_template_name_1> <index_template_name_2> <index_template_name_3>

Kibana Keystore Usage¶

The Kibana Keystore feature comes with a keybana-keystore tool, which permits to manage the settings in the keystore.

If your installation is a NetEye Cluster, you are advised to use kibana-keystore tool only from the cluster nodes where the Kibana resource is active.

Using the keybana-keystore tool from nodes where Kibana is not running will have no effect on the Kibana Keystore configuration.

El Proxy Security¶

The El Proxy module is automatically configured upon installation to use certificates for improved security and to comply with standard NetEye policies. This section describes some details of the configuration and can be used to modify the setup if required.

TLS configuration¶

The El Proxy server can start in HTTP or HTTPS mode; this is configured in the config web_server.tls section.

The available modes are:

None: The El Proxy server starts with TLS disabled. Example:
```
[web_server.tls]
type = "None"
```
PemCertificatePath: The El Proxy server starts with TLS enabled using the PEM certificates read from the local file system. When this method is used, the following information must be provided:
- certificate_path: path to the server public certificate
- private_key_path: path to the server private key
Example:
```
[web_server.tls]
type = "PemCertificatePath"
certificate_path = "/path/to/certs/ebp_server.crt.pem"
private_key_path = "/path/to/certs/private/ebp_server.key.pem"
```

Authentication to Elasticsearch¶

When the Elasticsearch client is created, the authentication method to be used to connect to Elasticsearch needs to be specified. The authentication method defined in the configuration file is only used for the serve command.

The available authentication methods are:

None: the client connects to Elasticsearch without authentication. Example:
```
[elasticsearch.auth]
type = "None"
```
BasicAuth: the client authenticates to Elasticsearch with username and password. When this method is used, the following information must be provided:
- username: name of the Elasticsearch user
- password: the password for the Elasticsearch user
```
[elasticsearch.auth]
type = "BasicAuth",
username = "myuser",
password = "mypassword"
```
PemCertificatePath: the client connects to Elasticsearch using the PEM certificates read from the local file system. When this method is used, the following information must be provided:
- certificate_path: path to the public certificate accepted by Elasticsearch
- private_key_path: path to the corresponding private key
Example:
```
[elasticsearch.auth]
type = "PemCertificatePath",
certificate_path = "/path/to/certs/ebp.crt.pem",
private_key_path = "/path/to/certs/private/ebp.key.pem",
```

El Proxy Verification¶

In El Proxy, the verify command can be used to understand if the underlying blockchain has been corrupted. In the remainder, some advanced aspects of the verification are outlined.

First Iteration Retrieval¶

During the verification process, one of the first steps performed by El Proxy is the retrieval of the first expected iteration. This aspect is particularly important when considering the retention policies, since the first expected iteration needs to be calculated based on the logs that we still expect to find in Elasticsearch. The following diagram outlines the first steps of the verification process which involve the retrieval of the first iteration and the various errors or warnings raised.

../_images/elproxy_blockchain_verification_first_iteration.png — Fig. 209 El Proxy verification process - First iteration retrieval.¶

(*) for more information please refer to the following Must Exists vs May Exist section

Must Exist vs May Exist

When retrieving the first expected iteration, we distinguish two possible cases:

El Proxy must find the first expected iteration in the blockchain to complete the verification successfully (Must Exist)

El Proxy may find the first expected iteration in the blockchain to complete the verification successfully (May Exist)

The difference between these two alternatives resides in how El Proxy is able to calculate the first expected iteration, taking into account that not all days have indexed logs. Given the first date within the retention period, El Proxy calculates the first expected iteration as follows:

It extracts the iteration of the first log indexed on the first date within the retention period. If this is not present because on that particular day no logs were indexed, El Proxy checks the following days, until the end of the blockchain, to understand which is the first indexed log within the retention and expects it to be present (Must Exist)

If no logs are found from the first date within the retention, El Proxy checks the previous days and it considers the first expected iteration to be the one of the last index log plus 1. In this case, El Proxy cannot be sure that a log with this specific iteration will be found, since no more logs could have been indexed after a specific date past to the first date within the retention (May Exist).

Warnings and Error Codes¶

The verification process may complete successfully, emit an error or a warning. Errors thrown by El Proxy lead to a failed verification, while warnings may appear also on successful completion.

The following table reports the warnings that could be reported by El Proxy during the verification process, along with the possible causes and actions that can be taken to address the issue.

Table 78 El Proxy Verification Warnings¶
Warning Code	Description	Actions
UntrustedFirstIterationNoStateHistory	The blockchain state history file was not found and therefore the first iteration was taken from Elasticsearch, leading to an untrusted first iteration.	After a successful verification, if the blockchain state history file was never written, no further actions are required. Otherwise, a manual investigation of the causes that brought to the absence of the blockchain state history file is required.
UntrustedFirstIterationInsufficientInfo	The blockchain state history file was found, but it does not contain enough information to assess the expected first iteration of the blockchain, which was thus retrieved from Elasticsearch and therefore it is untrusted.	This warning appears when the time between two consecutive verifications exceeds the retention period, leading to an outdated history and therefore to the retrieval from Elasticsearch. Increasing the frequency of the verifications will avoid the presence of the warning.
UnknownFirstIteration	El Proxy was unable to identify the first expected iteration, neither by fetching the Blockchain State History, nor by retrieving logs from Elasticsearch.	This warning appears if the blockchain is new and no logs have been indexed, a case in which no actions are required. As an alternative, if all logs have been deleted and this is the first verification, the history was not yet written, a case in which a manual investigation on the causes is necessary.
EmptyBlockchain	The first expected iteration and all the subsequent iterations were not found in the blockchain, thus the first expected iteration had been computed based on the last iteration present before the retention period of the blockchain.	This warning appears if no logs have been written within the retention period, requiring no further actions. In case you expected logs to be present, you should perform manual investigation to assess if all logs within the retention period might have been deleted.

Similarly, the following table reports the errors that could be reported by El Proxy during the verification process, along with the possible causes and actions that can be taken to address the issue.

Table 79 El Proxy Verification Errors¶
Error Code	Description	Actions
EmptyBlockchainCorruption	The blockchain is empty and the first expected iteration was not found, leading to a failed verification	There are mainly two causes that could lead to this error and in both cases a manual investigation of what has happened is required: the blockchain state history file contains wrong information, leading to a wrong expected first iteration, or all logs in the blockchain have been deleted
VerificationFailedError	A corruption was found in the blockchain and therefore the verification process failed	A manual investigation of the missing logs is required