Upgrading Search Guard

Across minor Elasticsearch releases

For a detailed overview on how to upgrade Search Guard across minor Elasticsearch releases, for example from 6.8.6 to 7.6.2, please refer to the documentation at https://docs.search-guard.com/7.x-40/upgrading .

The target Elasticsearch release must be supported by the Siren Federate plugin.

No security-related changes are required in the Siren Investigate configuration when you upgrade Search Guard across minor Elasticsearch releases.

Across major Elasticsearch releases

To perform a rolling restart and upgrade from 6.x to 7.x, where 7.x must be an Elasticsearch release that is supported by the Siren Federate plugin, your system must be running the following versions at a minimum:

  • Elasticsearch 6.8.6

  • Search Guard 6.8.x-25.0

If your system is running older versions of either Elasticsearch or Search Guard, please upgrade them before attempting to upgrade to 7.x .

The target Elasticsearch release must be supported by the Siren Federate plugin.

For a detailed overview on how to upgrade Search Guard across major Elasticsearch releases, for example from 6.8.6 to 7.6.2, please refer to the documentation at https://docs.search-guard.com/7.x-40/upgrading-6-7. A sample upgrade procedure for a single local node is described below to help you familiarize with the process before planning a production upgrade.

No security-related changes are required in the Siren Investigate configuration when you upgrade from Elasticsearch 6.x to Elasticsearch 7.x .

Example upgrade procedure

If you have a local installation of Elasticsearch version 6.8.6, Siren Federate, and Search Guard, use the following steps to become familiar with the upgrade process.

Step 1: Backing up the current security configuration

Before you begin, ensure that the administrative certificate and the CA bundle files for your local cluster are available on your local system.

  1. Download the latest Search Guard admin standalone version for Elasticsearch 7 from https://docs.search-guard.com/7.x-40/search-guard-versions and extract to a local directory named sgadmin.

  2. Change to the sgadmin directory and retrieve the security configuration from the running 6.8.6 node by running the sgadmin/tools/sgadmin.sh command as follows:

    tools/sgadmin.sh -nhnv -icl -h localhost -ff -prompt -cert "CN=sgadmin.crtfull.pem" -key "CN=sgadmin.key.pem" -cacert "chain-ca.pem" -backup "sgconfig6"

Use the following arguments in the command:

+ - -nhnv: Skips hostname verification. Can be omitted if certificates are set up to allow hostname verification. - -ff: Fail fast without retrying if there are issues with the configuration. - -icl: Accepts any cluster name. Can be changed by using -cn <cluster name> if you want the tool to check the name of the cluster before uploading the configuration. - -h: The hostname at which Elasticsearch is reachable. - -prompt: Tells sgadmin to prompt for the private key password. - -cert: The path to the administrative certificate file. - -key: The path to the file that contains the private key for the administrative certificate. - -cacert: The path to the file that contains the CA bundle. - -backup: The name of the directory where the configuration that is retrieved from the cluster will be stored.

If the command is successful, the following output is displayed:

Legacy index 'searchguard' (ES 6) detected (or forced). You should migrate the configuration!
See https://docs.search-guard.com/latest/upgrading-6-7 for more details.
Will retrieve 'sg/config' into ./sgadmin7/sgconfig/sg_config.yml (legacy mode)
   SUCC: Configuration for 'config' stored in ./sgadmin7/sgconfig/sg_config.yml
Will retrieve 'sg/roles' into ./sgadmin7/sgconfig/sg_roles.yml (legacy mode)
   SUCC: Configuration for 'roles' stored in ./sgadmin7/sgconfig/sg_roles.yml
Will retrieve 'sg/rolesmapping' into ./sgadmin7/sgconfig/sg_roles_mapping.yml (legacy mode)
   SUCC: Configuration for 'rolesmapping' stored in ./sgadmin7/sgconfig/sg_roles_mapping.yml
Will retrieve 'sg/internalusers' into ./sgadmin7/sgconfig/sg_internal_users.yml (legacy mode)
   SUCC: Configuration for 'internalusers' stored in ./sgadmin7/sgconfig/sg_internal_users.yml
Will retrieve 'sg/actiongroups' into ./sgadmin7/sgconfig/sg_action_groups.yml (legacy mode)
   SUCC: Configuration for 'actiongroups' stored in ./sgadmin7/sgconfig/sg_action_groups.yml

Step 2: Stopping the old Elasticsearch instance

You can now stop the old Elasticsearch instance and keep it as a backup; to stop the instance, press CTRL+C if it was started in a terminal or systemctl stop <your Elasticsearch unit name> if installed as a unit.

Step 3: Setting up the new Elasticsearch instance

  1. Download the OSS version of Elasticsearch 7.3.2 for your operating system from https://www.elastic.co/downloads/past-releases/elasticsearch-oss-7-3-2 and extract it.

  2. Download the latest Siren Federate release from https://siren.io/downloads/?product=siren-federate and extract it in a temporary directory.

  3. Copy the data folder from the old instance to the new. For example, run the following command:

rm -rf elasticsearch-7.3.2/data
cp -rp elasticsearch-6.8.6/data elasticsearch-7.3.2/data
  1. Copy all of the PKI files from the old instance to the new one. For example, run the following command:

cp -rp elasticsearch-6.8.6/config/*.jks elasticsearch-7.3.2/config/.
cp -rp elasticsearch-6.8.6/config/*.pem elasticsearch-7.3.2/config/.
cp -rp elasticsearch-6.8.6/config/*.p12 elasticsearch-7.3.2/config/.
  1. Copy over the Elasticsearch configuration file by running the following command:

cp elasticsearch-6.8.6/config/elasticsearch.yml elasticsearch-7.3.2/config/.
  1. Install Siren Federate and Search Guard by running the following command:

elasticsearch-7.3.2/bin/elasticsearch-plugin install -b file://<path to where you extracted the Federate archivive>/siren-federate-7.3.2-19.1-proguard-plugin.zip
elasticsearch-7.3.2/bin/elasticsearch-plugin install -b https://releases.floragunn.com/search-guard-7/7.3.2-37.0.0/search-guard-7-7.3.2-37.0.0.zip
  1. Start the node by running the following command:

bin/elasticsearch

Step 4: Migrating the saved Search Guard configuration

  1. To validate the Search Guard configuration that is saved in the sgconfig file, run the sgadmin command as follows:

    tools/sgadmin.sh -vc 6 -cd sgconfig6
  2. Migrate the configuration to the new format for Elasticsearch 7. To do so, create a new directory to store the upgraded configuration and run the sgadmin command as follows:

mkdir sgconfig7
tools/sgadmin.sh -nhnv -icl -h localhost -ff -prompt -cert "CN=sgadmin.crtfull.pem" -key "CN=sgadmin.key.pem" -cacert "chain-ca.pem" -migrate sgconfig7

Use the following arguments in the command:

  • -nhnv: Skips hostname verification, can be omitted if certificates are setup to allow hostname verification.

  • -ff: Fail fast if there are issues with the configuration without retrying.

  • -icl: Accepts any cluster name. Can be changed with -cn <cluster name> if you want the tool to check the name of the cluster before uploading the configuration.

  • -h: The hostname at which Elasticsearch is reachable.

  • -prompt: Tells sgadmin to prompt for the private key password.

  • -cert: The path to the administrative certificate file.

  • -key: The path to the file containing the private key for the administrative certificate.

  • -cacert: The path to the file containing the CA bundle.

  • -migrate: The path of an empty directory where the current cluster. configuration will be downloaded and migrated, then uploaded back to the cluster.

If the command is successful, the following output is displayed:

Legacy index 'searchguard' (ES 6) detected (or forced). You should migrate the configuration!
See https://docs.search-guard.com/latest/upgrading-6-7 for more details.
== Migration started ==
=======================
-> Backup current configuration to ./sgadmin7/sgconfig7
Will retrieve 'sg/config' into ./sgadmin7/sgconfig7/sg_config.yml (legacy mode)
   SUCC: Configuration for 'config' stored in ./sgadmin7/sgconfig7/sg_config.yml
Will retrieve 'sg/roles' into ./sgadmin7/sgconfig7/sg_roles.yml (legacy mode)
   SUCC: Configuration for 'roles' stored in ./sgadmin7/sgconfig7/sg_roles.yml
Will retrieve 'sg/rolesmapping' into ./sgadmin7/sgconfig7/sg_roles_mapping.yml (legacy mode)
   SUCC: Configuration for 'rolesmapping' stored in ./sgadmin7/sgconfig7/sg_roles_mapping.yml
Will retrieve 'sg/internalusers' into ./sgadmin7/sgconfig7/sg_internal_users.yml (legacy mode)
   SUCC: Configuration for 'internalusers' stored in ./sgadmin7/sgconfig7/sg_internal_users.yml
Will retrieve 'sg/actiongroups' into ./sgadmin7/sgconfig7/sg_action_groups.yml (legacy mode)
   SUCC: Configuration for 'actiongroups' stored in ./sgadmin7/sgconfig7/sg_action_groups.yml
  done
-> Migrate configuration to new format and store it here: ./sgadmin7/sgconfig7/v7
  done
-> Delete old searchguard index
Deleted index 'searchguard'  done
-> Upload new configuration into Elasticsearch cluster
./sgadmin7/sgconfig7/v7/sg_action_groups.yml OK
./sgadmin7/sgconfig7/v7/sg_internal_users.yml OK
./sgadmin7/sgconfig7/v7/sg_roles.yml OK
./sgadmin7/sgconfig7/v7/sg_roles_mapping.yml OK
./sgadmin7/sgconfig7/v7/sg_config.yml OK
./sgadmin7/sgconfig7/v7/sg_tenants.yml OK
Will update '_doc/config' with ./sgadmin7/sgconfig7/v7/sg_config.yml
   SUCC: Configuration for 'config' created or updated
Will update '_doc/roles' with ./sgadmin7/sgconfig7/v7/sg_roles.yml
   SUCC: Configuration for 'roles' created or updated
Will update '_doc/rolesmapping' with ./sgadmin7/sgconfig7/v7/sg_roles_mapping.yml
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '_doc/internalusers' with ./sgadmin7/sgconfig7/v7/sg_internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update '_doc/actiongroups' with ./sgadmin7/sgconfig7/v7/sg_action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Will update '_doc/tenants' with ./sgadmin7/sgconfig7/v7/sg_tenants.yml
   SUCC: Configuration for 'tenants' created or updated
Done with success
  done