Cluster Upgrade from 4.37 to 4.38¶
This guide will lead you through the steps specific for upgrading a NetEye Cluster installation from version 4.37 to 4.38.
During the upgrade, individual nodes will be put into standby mode. Thus, overall performance will be degraded until the upgrade is completed and all nodes are revoked from standby mode. Granted the environment connectivity is seamless, the upgrade procedure may take up to 30 minutes per node.
Warning
Remember that you must upgrade sequentially without skipping versions, therefore an upgrade to 4.38 is possible only from 4.37; for example, if you have version 4.27, you must first upgrade to the 4.28, then 4.29, and so on.
Breaking Changes¶
New Authentication Method¶
The authentication method has been updated to enable new features such as login in NetEye with federated identity providers through protocols such as OIDC (OAuth 2.0), SAML 2.0, or LDAP/Active Directory.
NetEye 4.38 introduces Keycloak as its access manager, enhancing security by enabling single sign-on (SSO) and role-based access control (RBAC) across NetEye’s advanced monitoring and threat protection features.
Keycloak is an open-source identity and access management (IAM) solution that provides authentication, authorization, and user management for modern applications. It is effectively the new hub for managing users and groups in NetEye.
This integration ensures that only authenticated and authorized users can access NetEye’s critical services, streamlining user management while maintaining robust security. Keycloak’s flexible integration with various identity providers further strengthens NetEye’s secure and unified monitoring environment.
More information about the new authentication method can be found in the Authentication section. This important change requires some additional steps to be taken before and after upgrading to the new version.
Upgrade of Elastic Stack to version 8.15¶
In NetEye 4.38, Elastic Stack is upgraded from version 8.14.3 to version 8.15.2.
To ensure the full compatibility of your installation, please review the official release notes, focusing in particular on the breaking changes of the Elastic Stack components.
Among the known issues and breaking changes listed by Elastic, we would like to emphasize those that may have more impact on NetEye installations than the rest:
input-snmp and input-snmptrap Logstash plugins are now bundled in the new Logstash SNMP integration plugin, so a migration may be needed
Prerequisites¶
Before starting the upgrade, you should read very carefully the latest release notes on NetEye’s blog and check out the features that will be changed or deprecated after the upgrade.
All NetEye packages installed on a currently running version must be updated according to the update procedure prior to running the upgrade.
NetEye must be up and running in a healthy state.
Disk Space required:
3GB for
/
and/var
150MB for
/boot
If the SIEM module is installed:
The rubygems.org domain should be reachable by the NetEye Master only during the update/upgrade procedure. This domain is needed to update additional Logstash plugins and thus is required only if you manually installed any Logstash plugin that is not present by default.
Cluster - Configuration of Keycloak PCS resource¶
In case you are upgrading a NetEye Cluster, before proceeding with the upgrade, you must:
Configure the Keycloak PCS resource located at
/usr/share/neteye/keycloak/cluster/Services-core-keycloak.conf
with the correct values in accordance with your cluster configuration.Confirm that the Keycloak PCS resource is correctly configured by enabling the switch Keycloak PCS file is configured located in
.
This is needed to ensure that the PCS resource will be correctly created during the upgrade process.
1. Run the Upgrade¶
The Cluster Upgrade is carried out by running the following command:
cluster# (nohup neteye upgrade &) && tail --retry -f nohup.out
Warning
If the SIEM feature module is installed and a new version of Elasticsearch is available, please note that the procedure will upgrade one node at the time and wait for the Elasticsearch cluster health status to turn green before proceeding with the next node. For more information, please consult the dedicated section.
After the command was executed, the output will inform if the upgrade was successful or not:
In case of successful upgrade you might need to restart the nodes to properly apply the upgrades. If the reboot is not needed, please skip the next step.
In case the command fails refer to the troubleshooting section.
2. Reboot Nodes¶
Restart each node, one at a time, to apply the upgrades correctly.
Run the reboot command
cluster-node-N# neteye node reboot
In case of a standard NetEye node, put it back online once the reboot is finished
cluster-node-N# pcs node unstandby --wait=300
You can now reboot the next node.
3. Cluster Reactivation¶
At this point you can proceed to restore the cluster to high availability operation.
Bring all cluster nodes back out of standby with this command on the last standard node
cluster# pcs node unstandby --all --wait=300 cluster# echo $?
0
If the exit code is different from 0, some nodes have not been reactivated, so please make sure that all nodes are active before proceeding.
Run the checks in the section Checking that the Cluster Status is Normal. If any of the above checks fail, please call our service and support team before proceeding.
Re-enable fencing on the last standard node, if it was enabled prior to the upgrade:
cluster# pcs property set stonith-enabled=true
4. Additional Tasks¶
New Authentication Method¶
After upgrading the new authentication method will be enabled along with the legacy one. This approach allows users to switch to the new authentication method at their own pace and test it before the final transition.
Login page with a greyed-out background indicates that you are using the legacy authentication method. Below the login form you will find a red button that will open the new login page.
After the upgrade, the management of users and groups should be handled through Keycloak rather than NetEye. Any users or groups created after the upgrade will need to be manually migrated to Keycloak.
At this point you need to log-in as a keycloak administrator and test the new login page as well as verifying that everything works as expected.
Note
By default only the root user has access to the Keycloak admin console. The password for the root user is saved in the file /root/.pwd_icingaweb2_root.
- We suggest to
Get inside the Keycloak admin console by visiting the
in the NetEye menuOnce inside, verify that the users are correctly imported from the legacy authentication method by clicking the
linkVerify that the groups are correctly imported from the legacy authentication method by clicking the
linkVerify the configuration of migrated LDAP/AD by clicking the
link, and then select the LDAP or Active Directory item in the listIf an LDAP/AD group backend was configured before the upgrade, verify in the
tab that a group-ldap-mapper type exists and that its configuration is correct
Verify roles defined in Keycloak and assigned to users and groups by visiting the
pageTest the login with different users and groups
During the upgrade process, users and groups are automatically migrated; however, we strongly recommend verifying that everything functions as expected.
Due to the complexity of the operation, it is possible that, depending on the current configuration, some errors may occur, leading to certain users or groups not being migrated correctly.
In such cases, manual migration may be necessary.
Warning
If errors occur during the automatic migration of LDAP/AD settings from NetEye to Keycloak, you will find all errors details in the log file /root/upgrade_4.38/failed_ldap_configuration.log
and you’ll have to correct the configuration after the upgrade as described in the Test your LDAP/AD configuration section.
Warning
If errors occur during the procedure of replicating admin permissions of any local/LDAP users in keycloak, you will find all errors details in log files located in /root/upgrade_4.38/
and you’ll have to migrate the permissions manually after the upgrade.
Note
We are only migrating admin permissions automatically during the upgrade, any other auth-related permissions will have to be migrated manually.
If something is not working as expected you can always use to the legacy login page for the time being, and adjust the configuration of the new authentication method by visiting the Authentication section for more information about the new authentication method.
page. Refer to theWarning
Before continuing with the definitive switch to the new authentication method, make sure that the login and the users are working as expected.
Until the new authentication method is switched as default, a new service check in the monitoring will be in critical in order to remind you that you are still using the legacy authentication method and the migration is not completed yet. The service check is called auth-finalization-check-neteyelocal and is part of the host NetEye Local Self Monitoring.
Warning
Until the new authentication method is switched as default, all the new users and groups created in
will also have to be created in the new authentication method in Configuration > Authentication.Once you are sure that everything is working as expected you can switch to the new authentication method by visiting the
page and enabling the switch Finalize Auth Migration. This step is required before upgrading to the next version.Warning
All Icingaweb2 roles (with permissions) will still be handled by Icinga Web 2 and will not be migrated or delegated to the new authentication method.
Warning
The switch to the new authentication method is irreversible and once you have switched to the new authentication method you cannot go back to the legacy one.
More information about the new authentication method can be found in the Authentication section.