Agents¶

The Elastic Beat feature¶

NetEye can receive data from Beats installed on monitored hosts (i.e., on the clients).

NetEye currently supports Filebeat as a Beat agent and the Filebeat NetFlow Module for internal use. Additional information about the Beat feature can be found in the official documentation.

The remainder of this section shows first how NetEye is configured to receive data from Beats, i.e., as a receiving point for data sent by Beats, then explains how to install and configure Beats on clients, using SSL certificates to protect the communication.

Overview of NetEye’s Beat infrastructure setup¶

Beats are part of the NetEye Elastic Stack module, which is an additional module, that can be installed following the directions in the NetEye Additional Components Installation section if you have the subscription.

Warning

Beats are intended as a replacement for Safed, even if they can coexist. However, since both Beat and Safed might process the same data, they would double the time and resources required, therefore it is suggested to activate only one of them.

The NetEye implementation allows Logstash to listen to incoming data on a secured TCP port (5044). Logstash then sends data into two flows:

to a file on disk, in the /neteye/shared/rsyslog/data folder, with the following name: %{[agent][hostname]}/%{+YYYY}/%{+MM}/%{+dd}/[LS]%{[host][hostname]}.log. The format of the file is the same used for safed files. This file is encrypted and its integrity validated, like it happens for Safed, and written to disk to preserve its inalterability.
to Elastic, to be displayed into preconfigured Kibana dashboards.

Communication is SSL protected, and certificates need to be installed on clients together with the agents, see next section for more information.

Note

When the module is installed there is no data flow until agents are installed on the clients to be monitored. Indeed, deployment on NetEye consists only of the set up of the listening infrastructure.

The Beat feature is currently a CLI-only feature: no GUI is available and the configuration should be done by editing configuration files.

Installation and Configuration of Beat Agents¶

Before being able to take fully advantage of the Beat feature, agents must be installed on the monitored hosts, along with the necessary certificates. On the hosts, any kind of Beat can be installed; for example, the Winlogbeat is available from the official download page; installation instructions are available as well. The agent configuration is stored in the YAML configuration file winlogbeat.yml. A description of the options available in the Beat’s configuration file can be found in the official documentation.

Note

You need to install a Beat whose version is compatible with the Elastic version installed on NetEye, which is 7.17. To find out which version of Beat you can install, please check the compatibility matrix

Relevant to the configuration are the following options:

ignore_older, which indicates how many hours/days it should gather data from. By default, indeed, the Beat collects all the data it finds, meaning it can act retroactively. This is the default option if not specified, so make sure to properly configure this option, to not overload the initial import of data and to avoid potential problems like crash of Logstash and ES disk space.
index: ”winlogbeat”, which is needed to match NetEye’s templates and ILM.

Elastic Agent¶

NetEye officially supports the implementation and the installation of Elastic Agents on NetEye Operative Node and Single Purpose Node types. Elastic Agent can be used to collect data from local and external sources with a single unified agent per host. More information can be found in the official Elastic-Agent documentation.

Elastic Agents in NetEye¶

NetEye adopts Elastic Agents in order to enhance the monitoring of different hosts, using available integrations given by the Elastic environment. A Fleet Server is implemented in the NetEye system in order to manage the different configurations and integrations of multiple Elastic Agents that can be added to the monitoring system. Make sure to respect all the Requirements in order to ensure the correct operation of all Elastic Agent integrations.

In order to manage the agents that are present in the NetEye system, two different policies will be applied:

NetEye Operative Nodes: This policy is applied to the agents running in the NetEye Operative nodes. It includes the following integrations:
- System: Collects authentication logs of the Operative nodes.
- Fleet Server: Coordinates all the enrolled Elastic Agents, updating and managing the policies across agents.
- APM Server: Receives performance data from internal and external applications.
NetEye Single-Purpose Nodes: This policy is assigned only to the Single-Purpose nodes of NetEye and includes the following integrations:
- System: Collects authentication logs of the Single-Purpose nodes.

Note

Although it is allowed to include integrations into NetEye-managed policies, any additional integrations must be carefully considered as they affect the computational load on the system.

Warning

The retention of monitoring logs and metrics collected by Elastic Agent is set to be unlimited by default, while data generated from the different integrations have specific retentions managed by Elastic.

Configuring Elastic Agent¶

Elastic Agent comes out-of-the-box in NetEye installations: every NetEye node hosts a running Elastic Agent with a preconfigured policy that depends on the role of the node in the NetEye system. Since the Elastic Agent implementation is officially supported, the Elastic Agents running on NetEye nodes should be assigned to the default NetEye policies and not to other custom policies.

In order to customize the policy parameters and the NetEye-managed integrations with the -neteye suffix:

Customize the configuration files in /neteye/local/elastic-agent/conf/fleet/templates/ specifying the parameters you want to change.
Apply the new configuration on all the NetEye nodes by running the following command on the node where the configuration file has been modified:
```
neteye tenant config apply master
```
This will synchronize the changes in the configuration file and apply the customization to the enrolled Elastic Agents in the system.

Fleet Server¶

By default, Fleet Server is exposed, through Nginx, using the certificate and key located at /neteye/shared/nginx/conf/certs/fleet-server-external.crt.pem and /neteye/shared/nginx/conf/certs/private/fleet-server-external.key.

The certificate and key are automatically generated upon NetEye first installation procedure, however it is recommended to install a new trusted certificate as soon as possible.

Warning

Please take note that in order to allow the correct communication between Elastic Agents and the Fleet Server, the full chain of certificates should be included in the certificate file.

Moreover, the cryptographic settings applied to the SSL connections replicate the settings applied at a system level, which may be changed as described in the official RHEL documentation.

Elastic Agent on Satellites¶

The installation of Elastic Agents on NetEye Satellites is not yet officially managed by the NetEye system. Elastic Agent can still be installed and configured manually in order to obtain a Tenant-related Agent.

The following guidelines will explain how to correctly configure an Elastic Agent on a NetEye Satellite:

Before adding the Elastic Agent to the NetEye system, a dedicated Policy is needed for the Satellite: Through the “Fleet” section in Kibana, an additional Policy called <tenant_display_name> Satellite can be created, using <tenant_id> as namespace for the Policy and the associated integrations.
After the installation of Elastic Agent from the NetEye repositories has been completed, the Agent can be enrolled to Elastic using the Fleet Server of the NetEye Master, reachable at <neteye_master_fqdn>:8220.
The Elastic Agent can use the already configured Output neteye-external-output, connected to the Elasticsearch server on the NetEye Master.
Once the Elastic Agent has been enrolled in the NetEye Fleet, additional integrations can be added and applied to the Tenant-related Elastic Agent running on the NetEye Satellite. Every integration belonging to the previously created Policy, associated with a specific Tenant, should have the -<tenant_id> suffix in order to maintain a good separation among different Tenants.

For example, a good choice for the APM integration of a Tenant having tenant_id = 12345 could be apm-12345

Signing Elastic Agent Logs¶

In order to sign documents collected by any Elastic Agent in the NetEye environment using El Proxy, it is possible to configure Elastic Agent to send logs to Logstash for the signing process. Once El Proxy has been correctly configured and enabled as described in Enabling El Proxy, the subsequent sections can be followed in order to apply the correct configuration to the specific scenario:

Note

If your Elastic Agents need to connect to the NetEye Logstash using an hostname different than the NetEye Hostname (declared in /etc/neteye-cluster for Clusters and hostname --fqdn for Single Nodes) in the case they run outside the company network, for example, you can substitute the certificate /neteye/shared/logstash/conf/certs/logstash-server-external.crt.pem and private key /neteye/shared/logstash/conf/certs/private/logstash-server-external.key with trusted certificates valid for the needed hostname.

Elastic Agent on the Master Tenant¶

If Elastic Agent belongs to the Master Tenant, it is enough to change the output of the Elastic Agent and send the logs to be signed to Logstash instead of sending them directly to Elasticsearch.

This operation can be executed under Fleet / Agent Policies / <selected_policy> / Settings by changing the Output for integrations field to the preconfigured neteye-external-logstash-output output.

Note that <selected_policy> should be the policy assigned to the Elastic Agent that wll then send the logs to be signed by El Proxy. Every other Elastic Agent connected to the same policy will change its output, starting to send logs to Logstash.

Elastic Agent on other Tenants¶

In the case Elastic Agent belongs to a Tenant different to the Master Tenant, the Elastic Agent should send the collected logs to the Logstash instance running on the NetEye Satellite of its Tenant.

In order to change the output of the Elastic Agent that should send logs to be signed, an additional output should be created in Fleet / Settings / Outputs defining as Logstash the output type of the configuration and filling the other parameters of the form.

Since the documents should be sent to the Logstash instance of the NetEye Satellite, the Satellite FQDN and the port 5045 should be specified as Logstash Host, for example <satellite_fqdn>:5045.

Note that the NetEye instance of Logstash already has a preconfigured pipeline dedicated to Elastic Agent input sources, no additional configurations are required in Logstash.

Once the new output options has been stored in Fleet, it is possible to assign the newly created output to the policy of the Elastic Agent that is collecting logs to be signed.

Filebeat Netflow module specific configuration¶

In NetEye, Filebeat is shipped with the NetFlow Module included. The module should be configured without directly modifying the configuration file. Any change in the configuration file will be overwritten during future updates and result in the reactivation of the module or losing particular customizations.

The correct configuration can be accomplished by setting the following three environment variables in the /neteye/shared/filebeat/conf/sysconfig/filebeat-user-customization environment file:

NETFLOW_ENABLED: which can be used to enable or disable the listening of logs on the specified host and port. Default: true
NETFLOW_HOST: The host to which the module will listen on. Default: 0.0.0.0 (all hosts)
NETFLOW_PORT: The port to listen on. Default: 2055

After changing the value of any of the aforementioned variables, the neteye install command should be executed to apply the changes:

neteye# neteye install

Warning

If the module is deactivated using the filebeat modules disable command and not using the associated environment variable, Filebeat will rename the configuration file and, at the next update, the module will be re-installed and re-activated.

Use of SSL certificates¶

Server certificates of Logstash allowing communication with Beats must be stored in the /neteye/shared/logstash/conf/certs/ directory, with names logstash-server.crt.pem and private/logstash-server.key. Additionally, also the root-ca.crt certificate must be available in the same directory.

The structure mentioned above for the certificates must be organised as:

certs/
   ├── logstash-server.crt.pem
   ├── root-ca.crt
   └── private/
          └── logstash-server.key

The certificates are stored under the logstash configuration directory, because it is indeed Logstash that listens for incoming Beat data flows.

As a consequence, all Beat clients must use a client certificate to send output data to Logstash. Please refer to the Elastic official documentation, for example the Filebeat SSL configuration is available here.

An example of Filebeat to Logstash SSL communication configuration is the following:

#--------- Logstash output ------------------------------------
    output.logstash:
      # The Logstash hosts
      hosts: ["yourNetEyeDomain.example:5044"]

      # List of root certificates for HTTPS server verifications
      ssl.certificate_authorities: ["/root/beat/root-ca.crt"]

      # Certificate for SSL client authentication
      ssl.certificate: "/root/beat/logstash-client.crt.pem"

      # Client Certificate Key
      ssl.key: "/root/beat/private/logstash-client.key.pem"

Self-signed certificates¶

Note

For production systems, you should upload your own certificates on NetEye. Moreover, you should use your own certificates also for all Beat clients. Self-signed certificates must never be used on production systems, but only for testing and demo purposes.

Self-signed certificates (logstash-server.crt.pem and private/logstash-server.key) and the Root CA (root-ca.crt) are shipped with NetEye for Logstash. Self-signed certificates for Beat clients can be generated from the CLI as follows:

you can run the script usr/share/neteye/scripts/security/generate_client_certs.sh using three suitable parameters:

The client name
The common name (CN) and information for the other certificate’s field
The output directory

An example of command line is the following:

/bin/bash /usr/share/neteye/scripts/security/generate_client_certs.sh \
    logstash-client \
    "/CN=logstash-client/OU=client/O=client/L=Bolzano/ST=Bolzano/C=IT" \
    "/root/beat/"

Inputs configuration¶

To set customer-specific filebeat inputs you can add a file with .yml extension in the directory /neteye/shared/filebeat/conf/inputs.d/. Configuration will be read and applied from .yml files only: any file with different extension will be ignored. To maintain a custom configuration saved but disabled, you should rename the file with a different extension, for example mqtt.yml can be disabled by renaming it to mqtt.yml.disable.

A sample configuration can be found in file /neteye/shared/filebeat/conf/inputs.d/mqtt.yml.sample.