Cluster Upgrade from 4.24 to 4.25¶
This guide will lead you through the steps specific for upgrading a NetEye Cluster installation from version 4.24 to 4.25.
Warning
Remember that you must upgrade sequentially without skipping versions, therefore an upgrade to 4.24 is possible only from 4.23; for example, if you have version 4.21, you must first upgrade to the 4.22, then 4.23, and so on.
Before starting an upgrade, you should very carefully read the latest release notes on NetEye’s blog and check the feature changes and deprecations specific to the version being upgraded. You should check also the whole section Breaking Changes below.
The remainder of this section is organised as follows. Section Breaking Changes introduces substantial changes that users must be aware of before starting the upgrade procedure and may require to carry out some tasks before starting the upgrade; section Prerequisites provide information to be known before starting the upgrade procedure; section Conventions Used defines some notation used in this procedure; section NetEye Single Node Upgrade Procedure presents the actual procedure, including directions for special nodes; section Cluster Reactivation instructs on how to bring the NetEye Cluster back to complete functionality, and finally section Additional Tasks shows which tasks must be executed after the upgrade procedure has been successfully executed.
Breaking Changes¶
NetEye Setup¶
Hostname Validation
From NetEye 4.25 onwards, hostname will be validated with follow specifications during neteye_secure_install. Existing NetEye installations with hostname that are not compliant with these specifications will only get a warning during neteye_secure_install, but do not need to change the hostname:
Only letters allowed in the first position
Hostname must consist only of letters [A-Za-z] numbers [0-9] hyphens - and dot
Hostname must end with a letter [A-Za-z] or a number [0-9]
Hostname must not contain two consecutive dots
Hostname must not contain two hyphens in third and fourth position:
valid hostname: neteye-test-01, neteye-production, neteye-node1, master, master.lan,
invalid hostname: ne–teye, -neteye-, neteye_01, 3neteye, @neteye, neteye-.com, master..lan
Tags and new neteye node tags command
Release 4.24 introduced RHEL 8 as new underlying operating system together with automatic registration and Red Hat Insights.
This means that, if the NetEye Single Node, Cluster, or Satellite has not yet been registered to Red Hat using the neteye node tags set command, the upgrade process will temporarily stop and ask for a few data (Customer ID, contract number, type of installation and deployment, see Section neteye node tags set for more information).
Make sure to have all these data at hand during the upgrade process; if you do not know some or all of these data, please refer to the official channels: sales, consultants, or support portal to receive them.
Elastic Stack¶
Kibana customization
The 4.25 version of NetEye updates Elastic Stack to version 7.17.4. In order
to address a deprecation warning dispatched by Kibana, the update renames a variable in
the /neteye/shared/kibana/conf/sysconfig/kibana file. In order to simplify future
updates that could involve it, therefore minimising the users’ effort, the file is not
set as a config file anymore. This change will generate a /neteye/shared/kibana/conf/sysconfig/kibana.rpmsave
file and, in case custom options were present, these must be migrated to the
/neteye/shared/kibana/conf/sysconfig/kibana-user-customization file.
More details about the change and how to migrate custom options can be found in Section User Customization.
Elasticsearch Index Templates
From version 4.25, NetEye installs and uses the new Index Templates of Elasticsearch, which configure the settings of the indices generated by El Proxy. These Index Templates are in the new composable index template format, which by design completely override all Legacy Index Templates that match the same indices.
In simple terms, the newly introduced templates match the indices with pattern
filebeat-7.17.4-elproxysigned-*. So if you have any custom Legacy Index Template
that need to be applied on these indices, you need to migrate them to the new composable
index templates. Please refer to the Section Customizing the Composable Index Templates
to perform the migration.
Moreover, from NetEye 4.25 onwards in case you need to create additional Index Templates for any Beats agents, you must follow the procedure described in Section Generating Templates for Additional Beats. Not following this procedure will result in El Proxy indices having wrong ILM policies and index mappings.
Prerequisites¶
Upgrading a NetEye Cluster will take a nontrivial amount of time. During the upgrade, individual nodes will be put into standby mode and so overall performance will be degraded until the upgrade procedure is completed and all nodes are removed from standby mode.
An estimate for the time needed for a full upgrade (update + upgrade) when the cluster is healthy, there is no additional NetEye modules installed, and the procedure is successful is approximately 30 minutes, plus 15 minutes per node.
So for instance on a 3-node cluster it may take approximately 1 hour and 15 minutes (30 + 15*3).
Warning
This estimate does not include the time required to download the packages and for the manual intervention: migration of configurations due to breaking changes, failure of tasks during the execution of the neteye update and neteye upgrade commands.
Conventions Used¶
A NetEye cluster can be composed by different types of nodes, including Elastic-only and Voting-only nodes, which require a different upgrade procedure. Therefore, the following notation has been devised, to identify nodes in the cluster.
(ALL) is the set of all cluster nodes
(N) indicates the NetEye master node of the Cluster
(E) is an Elastic-only node
(V) is a Voting-only node
(OTHER) is the set of all nodes excluding (N), (E), and (V)
For example if we take the sample cluster defined in
The Elected NetEye Master, (ALL) is my-neteye-01,
my-neteye-02, my-neteye-03, my-neteye-04, and
my-neteye-05.
(N) is
my-neteye-01(OTHER) is composed by
my-neteye-02andmy-neteye-03(E) is
my-neteye-04(V) is
my-neteye-05
Note
Please see The Elected NetEye Master for a discussion about the Cluster Master Node.
Running the Upgrade¶
Recall that if you still did not provide tags for registration to Red Hat and Red Hat Insights, the upgrade process will be stopped to allow you to set them. Refer to Section Breaking Changes for more information.
The Cluster Upgrade is carried out by running the command:
cluster# (nohup neteye upgrade &) && tail --retry -f nohup.out
All the tasks carried out by the command are listed in section neteye upgrade; a dedicated section provides directions in case the command fails.
Warning
The neteye upgrade command can be run on a standard NetEye node, but in must be never issued on an Elastic-only (E) or a Voting-only (V) Node, because it would turn these nodes into NetEye Nodes.
Special Nodes¶
In the context of the Upgrade procedure, special nodes are Elastic-only (E) and Voting-only (V) Nodes. They do not need to be upgraded manually, because the neteye upgrade command will automatically take care of upgrading them.
Additional Tasks¶
In this upgrade, no additional manual step is required.
Cluster Reactivation¶
You can now restore the cluster to high availability operation.
Bring all cluster nodes back out of standby with this command on the last node (N):
# pcs node unstandby --all --wait=300 # echo $?
0If the exit code is different from 0, some nodes have not been not reactivated, so please be sure that all nodes are active before proceeding.
Run the checks in the section Checking that the Cluster Status is Normal. If any of the above checks fail, please call our service and support team before proceeding.
Re-enable fencing on the last node (N), if it was enable prior to the upgrade:
# pcs property set stonith-enabled=true
