Setting up SAML and Okta on Elasticsearch with X-Pack

To set up this up you must do the following:

Configure Okta.
Configure Elasticsearch.
Configure Investigate.
Test your login.

Prerequisites

Ensure that you have the following folders:

ELASTICSEARCH_BASE_DIR
INVESTIGATE_BASE_DIR
TEMP_FOLDER

Configuring Okta

You need one administrator account. If you do not have an Okta account you can register for a 30-day trial. Ensure you register using an email account you have access to because you need to verify the email account.

Save your OKTA-TRIAL-NUMBER as you will have to replace it in URLs. For example, the URL to your Okta admin console should be in the following form:

https://trial-<OKTA-TRIAL-NUMBER>-admin.okta.com/admin/getting-started

Keep two different browsers open:

One where you are logged in to Okta as the user that registered for the trial account.
One in which you try the SAML login flow once all components are configured.

Creating users

In the Directory section create the users required for this demo setup. In this tutorial we use users with emails at siren.io sirenuser@siren.io and sirenadmin@siren.io.

Create okta users

Creating groups

Create the required groups. Give the groups an okta_ prefix to make it easier to distinguish them later, such as: okta_siren_all_group, okta_siren_user_group, and okta_siren_admin_group.

Create okta groups

Assigning users to groups

Click the group name.
Click Assign people.
Click the plus icon.
Assign the users to groups as follows:

sirenadmin → `okta_siren_all_group, okta_siren_user_group, okta_siren_admin_group `

sirenuser → `okta_siren_all_group, okta_siren_user_group `

Creating the Investigate application

Click Create App Integration.
Click SAML 2.0.
Click Next.
In App name enter investigate.
Click Next.
In Single sign-on URL, enter http://localhost:5606/investigate-access-control/saml/callback
In Audience URI (SP Entity ID), enter siren-saml.
Add all values like on the below screenshot.
In Single Logout URL, enter http://localhost:5606/investigate-access-control/saml/logout

Generate private key and cert for Elasticsearch.

cs TEMP_FOLDER
openssl req -x509 -newkey rsa:2048 -keyout saml-sign.key -out saml-sign.crt -days 365 -nodes -subj "/CN=elasticsearch"

this should generate two files saml-sign.cert and saml-sign.key

Upload saml-sign.cert to the OKTA app advanced settings. Note: These files will be needed to configure Elasticsearch later.
Check that the upload was successful. You should see the following:
Click Next and then select This is an internal app that we have created.
Click Finish.

Assigning users to the app

Assign users to Investigate app in OKTA

Assigning groups to the app

Assign groups to Investigate app in OKTA

Fetching idp metadata

Go to Investigate → Sign On tab.
Fetch the metadata XML file and save it in TEMP_FOLDER as okta-idp-metadata.xml.

Fetch idp metadata in Okta

It will look something like the following:

Idp metadata example in Okta

We will configure Elasticsearch to read the metadata from file (not from the URL) every time you make a change in the app configuration you have to re-fetch and re-save this file.

Configuring attributes

Configure additional attributes which are used when matching:

okta users → siren users

siren users → groups

In the OKTA interface they are called Attribute statements and Group attribute statements.

Click Application → Investigate → SAML Settings → Edit.

Configure attributes in OKTA

Configuring Elasticsearch

Execute the following command:

mkdir <ELASTICSEARCH_BASE_DIR>/config/saml
cp <TEMP_DIR>/okta-idp-metadata.xml <TEMP_DIR>/saml-sign.crt <TEMP_DIR>/saml-sign.key <ELASTICSEARCH_BASE_DIR>/config/saml/
cd <ELASTICSEARCH_BASE_DIR>
bin/elasticsearch-plugin install -b https://download.support.siren.io/federate/8.15.5-37.2.zip

Edit elasticsearch.yml

cluster.name: siren-distribution
node.name: siren-node
ingest.geoip.downloader.enabled: false
xpack.ml.enabled: false
discovery.type: single-node
indices.query.bool.max_clause_count: 15000
indices.breaker.total.use_real_memory: false
cluster.routing.allocation.disk.watermark.low: 3gb
cluster.routing.allocation.disk.watermark.high: 2gb
cluster.routing.allocation.disk.watermark.flood_stage: 1gb

siren.dataspaces.jws.pubkey: 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3CVFpHclqIEEERUhC/oDB+8ib+nDGIcmrIppGmmsmj5K65XCCZmHilR2fx9CNbjR5WLQDeOT0zhK0FHip41qNbOGwLbz1ehpquEwR/+W6bKyWkk0wMoS1npVSIZL0QygjatIszjKIgOti2FEzPxFxmHoroVluRTm84oIuY+KX/ExxLbS69ORJ8MUFu5oc/B1lNSmWwCCQlJmjjKR4RJt81otQkRQ+tMQS6Aek/bQWz23HhMBypwdXbXlZsLOO4ZKTWQrIZ1F32c+/6+mnvW2UKCISSLucKWBj3/sIMoKXKfFjsfyb41O9DXO3BZcQkWvw98mRhiZ/VzPUz6nwJd3UwIDAQAB'

xpack.security:
  enabled: true
  authc:
    token.enabled: true
    realms:
      native:
        native1:
          order: 0
      saml:
        saml1:
          order: 2
          nameid_format: "urn:oasis:names:tc:SAML:2.0:nameid-format:persistent"
          attributes:
            principal: "username"
            groups: "groups"
          idp:
            metadata.path: "<ELASTICSEARCH_BASE_DIR>/config/saml/okta-idp-metadata.xml"
            entity_id: "http://www.okta.com/exkptz4673Bx3aWgz697"  # <-- this id MUST match the entityId property from okta-idp-metadata.xml
          sp:
            entity_id: "siren-saml" # <-- must match Audience Restriction in okta
            acs: "http://localhost:5606/investigate-access-control/saml/callback"
            logout: "http://localhost:5606/investigate-access-control/logout"
          signing:
            certificate: "<ELASTICSEARCH_BASE_DIR>/config/saml/saml-sign.crt"
            key: "<ELASTICSEARCH_BASE_DIR>/config/saml/saml-sign.key"

Start Elasticsearch and configure basic users:

./bin/elasticsearch

./bin/elasticsearch-setup-passwords interactive

Answer password for all users.

Activate the trial license. Without it, SAML realm configuration will NOT work.

curl -uelastic:password -XPOST "http://localhost:9200/_license/start_trial?acknowledge=true"

{"acknowledged":true,"trial_was_started":true,"type":"trial"}

Stop and start Elasticsearch:
```
CTRL+C
./bin/elasticsearch
```

Install Investigate users and roles:

chmod 755 <INVESTIGATE_BASE_DIR>/docs/modules/siren-investigate/attachments/xpack/full/initroles.sh
chmod 755 <INVESTIGATE_BASE_DIR>/docs/modules/siren-investigate/attachments/xpack/initusers.sh

<INVESTIGATE_BASE_DIR>/docs/modules/siren-investigate/attachments/xpack/full/initroles.sh
# here answer only to second question
<INVESTIGATE_BASE_DIR>/docs/modules/siren-investigate/attachments/xpack/initusers.sh
# here answer only to second question

Install the Investigate license.

curl -XPUT -uelastic:password -H 'Content-Type: application/json' -T investigate-license http://localhost:9200/_siren/license

Add role mappings to map keycloak grouls to the Elasticsearch roles:

curl -k -XPUT -uelastic:password http://localhost:9200/_security/role_mapping/investigate_okta_admin -H "Content-Type: application/json" -d '{
  "roles": [ "investigate_admin" ],
  "enabled": true,
  "rules": { "all": [
        { "field": { "realm.name": "saml1" } },
        { "field": { "groups": "okta_siren_admin_group"} }
  ] }
}'

curl -k -XPUT -uelastic:password http://localhost:9200/_security/role_mapping/investigate_okta_user -H "Content-Type: application/json" -d '{
  "roles": [ "investigate_user" ],
  "enabled": true,
  "rules": { "all": [
        { "field": { "realm.name": "saml1" } },
        { "field": { "groups": "okta_siren_user_group"} }
  ] }
}'

Configuring Investigate

Create a file called <INVESTIGATE_BASE_DIR>/config/investigate.pem and put the following content in it:

Edit config/investigate.yml as follows:

elasticsearch.url: "http://localhost:9200"
elasticsearch.username: "sirenserver"
elasticsearch.password: "password"
server.ssl.enabled: false
elasticsearch.ssl.verificationMode: none
server.host: "0.0.0.0"
shared_index_components:
  enabled: true
  dataspaceSignature:
    algorithm: 'RS256'
    privateKey:
      key: 'config/investigate.pem'
      passphrase: 'password'

logging.promoteDebug:
  tags: [] #'samlAuth', 'samlLogin', 'samlLogout'
  toLevel: 'info'

investigate_access_control:
  enabled: true
  acl:
    enabled: true
  admin_role: investigate_admin
  session:
    backend: false
  cookie:
    name: 'kac'
    password: '12345678123456781234567812345678'
    secure: false
  backend: xpack
  backends:
    xpack:
      authenticator: saml
      saml:
        realm: saml1

Start investigate:
```
./bin/investigate
```

Testing login

Go to localhost:5606. You should be redirected to keycloak.
Login with sirenadmin password. You should be redirected back to Investigate where you should see yourself logged as sirenadmin mapped to proper es roles.

When logging in for the first time, you might have to setup 2-factor authentication. The following are example screens when doing this for sirenuser