Elasticsearch security helper tool

The secure communication provided by the X-Pack Security requires additional parameters such as authentication certificates to interact with the Elastic Stack APIs. We have developed a few helper tools, based on curl, to simplify your interaction with the APIs.

The Elasticsearch helper script lets you omit all the authentication parameters for the admin user, which would otherwise be required.

Location: /usr/share/neteye/elasticsearch/scripts/es_curl.sh

The NetEye helper script can be used instead if you only need read permission for the fields @timestamp and host on the Logstash index entries. This script is used by NetEye for self-monitoring activities.

Location: /usr/share/neteye/elasticsearch/scripts/es_neteye_curl.sh

Elasticsearch System indices on Single Nodes

Elasticsearch System indices are a type of indices meant to be used only internally by the components of the Elastic Stack. They may be associated with some System template which specifies their settings.

Some System templates may require System indices to have at least one replica, which in simple terms means that the index must be replicated on more than one node (note that this may apply also for non-System templates and indices). While having replicas is straightforward on a NetEye Cluster, on Single Node installations this is not possible, which causes these indices to go in yellow state due to the so-called unassigned shards warning.

Since on Single Node installations this warning cannot be solved by assigning shards to other nodes, the solution to the problem is to tell Elasticsearch that non-replicated indices should be allowed.

NetEye applies this solution by default during the neteye install on the templates and indices which have this problem. However, some templates and indices are created only when some features of the Elastic Stack are triggered by the user and must be manually fixed by the Elastic Stack administrator.

To facilitate the job of admin though, NetEye provides a set of scripts which help to fix the problem by setting the option index.auto_expand_replicas to 0-1 on the templates allowing the indices to have zero replicas on Single Node installations.

Depening whether the problematic index is managed by Fleet or not, you may use the appropriate script.

Index Templates Managed by Fleet

To fix index templates managed by Fleet (for example those that are automatically created when installing a new Elastic integration) you can use the following script that will resolve this issue on all the Fleet-managed index templates and already created associated indexes.

neteye# python3 /usr/share/neteye/elasticsearch/scripts/configurator/fix_fleet_integrations_autoexpand_replicas.py

Index Templates Not Managed by Fleet

Suppose that, after the creation and execution of a Security rule, you identified that the Elasticsearch template that causes the index to have the number of replicas set to one (or more) is named .items-default. You can then call the script as follows:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh ".items-default"

Note

The script works only with the composable index templates introduced in Elasticsearch 7.8 and does not support the Legacy index templates.

Moreover, the script supports the update of multiple Index Templates at once. To perform such operation simply pass the multiple Index Templates names as arguments, like this:

neteye# bash /usr/share/neteye/elasticsearch/scripts/elasticsearch_set_autoexpand_replicas_to_index_templates_and_indexes.sh <index_template_name_1> <index_template_name_2> <index_template_name_3>