OpenID Connect authentication support

This section offers an overview of how to integrate Siren Investigate with the OpenID Connect authentication support provided by Search Guard Classic.

Prerequisites

Before you begin, ensure that you have completed the following steps:

Set up security certificates
Integrate Search Guard Classic

When a user opens an OpenID Connect-enabled instance of Siren Investigate, they will be redirected to the Identity Provider (IdP) to enter their credentials. The IdP then generates an access token that is used by Siren Investigate to retrieve an ID token. Both tokens are encoded by using JSON and are signed with a public key.

The fields in the ID token (claims) are then parsed to retrieve the following information:

a unique user identifier (username, subject name or email address);
the roles granted to the users by the Identity Provider (backend roles).

The configuration defined in sg_role_mappings.yml is then used to map the backend roles to one or more Elasticsearch roles (the ones defined in sg_roles.yml).

In order to validate tokens, both Investigate and Elasticsearch will need to fetch the public keys from the Identity Provider’s discovery URL; this URL ends with /.well-known/openid-configuration and can usually be retrieved from the configuration UI of the provider.

Before proceeding with the configuration, you will need the following information:

the discovery URL of your Identity Provider;
the name of the ID token claim that contains the unique user identifier;
the name of the ID token claim that contains the backend roles.

Search Guard Classic configuration

In order to configure Search Guard Classic to work with an OpenID Connect compliant identity provider, you’ll need to add an authentication domain to the sg_config.yml file alongside the existing domains defined in the authc section.

An OpenID Connect authentication domain requires the following configuration options:

subject_key: the name of the ID token claim that contains the unique user identifier;
roles_key: the name of the ID token claim that contains the user’s roles. The claim can either be an array of strings or a string with a comma separated list of values.
openid_connect_url: the discovery URL of the Identity Provider.
enable_ssl: usually set to true to enable SSL for connections to the Identity Provider.
verify_hostnames: set to true to verify the hostname of the Identity Provider. Should be set to false only in development environments where the IdP is configured with a self-signed certificate that does not match its host name.

You will also need to ensure that challenge is disabled in the http_authenticator section of the basic_internal_auth_domain domain.

For additional options please refer to the Search Guard Classic documentation; a reference example follows:

_sg_meta:
  type: "config"
  config_version: 2

sg_config:
  dynamic:
    respect_request_indices_options: true
    do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      # The standard authentication domain using the internal user database
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: true
        # Basic authentication is attempted first
        order: 0
        http_authenticator:
          type: basic
          # Challenge is disabled
          challenge: false
        # User information is retrieved from the internal database
        authentication_backend:
          type: internal
      # The additional OpenID connect authentication domain
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        # OpenID authentication is attempted if basic authentication fails
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            # The name of the ID token claim that contains the unique user identifier
            subject_key: sub
            # The name of the ID token claim that contains the user's roles
            roles_key: roles
            # The discovery URL of the Identity Provider
            openid_connect_url: https://idp.local/auth/realms/master/.well-known/openid-configuration
            enable_ssl: true
            verify_hostnames: true
        # User information is provided in tokens so there's no need for any
        # authentication backend.
        authentication_backend:
          type: noop

The order attribute of the openid_auth_domain should be higher than the order specified for the basic_internal_auth_domain as otherwise all basic auth request will trigger a warning every time Investigate checks the connectivity to Elasticsearch.

After you edit the sg_config.yml file, upload it to the cluster by using sgadmin.

Configuring Siren Investigate

To enable OpenID Connect support in Siren Investigate, you will need to set the following variables under the investigate_access_control section:

public_url: the URL used by end users to open Investigate, including the base path; for example, if Investigate is running behind a reverse proxy at https://investigate.local/investigate, public_url must be set to https://investigate.local/investigate.
backends.searchguard.authenticator: set to openid.
backends.searchguard.openid.discovery_url: set to the same discovery URL specified in the Search Guard Classic configuration.
backends.searchguard.openid.client_id: set to the OpenID Connect client ID configured in the Identity Provider.
backends.searchguard.openid.client_secret: set to the password of the client_id specified in the previous step.
backends.searchguard.openid.scope: an array of additional scopes to request. Some identity providers might require the profile scope to return user information in the ID token, most will require the offline_access scope to return a refresh token.
backends.searchguard.openid.refresh_interval: if set to a number of seconds greater than 0, Investigate will refresh its access token and refresh user information periodically to keep the session alive. If not set, the Investigate user session will terminate once the access token expires. Should be lower than the access token lifetime.
session.backend: by default, Investigate will store user tokens in an encrypted browser cookie, however this might block the initiation of a session if the tokens are too large to be contained in a 5K cookie. When setting this option to true, Investigate will store the tokens in memory and put only a reference to the session in an encrypted browser cookie. If you are using multiple Investigate instances behind a reverse proxy and the option is enabled, make sure that session affinity is enabled when using this option, as the individual Investigate instance won’t share backend sessions between them.

A minimal configuration follows:

investigate_access_control:
  enabled: true
  public_url: "https://investigate.local/investigate"
  acl:
    enabled: true
  admin_role: investigate_admin
  session:
    backend: false
  cookie:
    name: 'kac'
    password: '12345678123456781234567812345678'
    secure: true
  backend: searchguard
  backends:
    searchguard:
      authenticator: openid
      openid:
        discovery_url: "https://idp.local/abcd/.well-known/openid-configuration"
        client_id: "123-abc-123"
        client_secret: "secret"

Restart Investigate after applying the changes to the investigate_access_control section.

If the configuration is correct, when opening Investigate you should be redirected to the Identity Provider login page and after entering your credentials be redirected to Investigate and logged in.

When logged in, the user information tooltip should:

list the expected Search Guard Classic roles (displayed as Elasticsearch roles)
list all the roles declared in the ID token roles claim as DLS roles.

User information

Office 365 / Azure Active Directory configuration

In order to enable logging into Investigate with Office 365 / Azure Active Directory credentials, you will need to:

decide how Active Directory groups will be mapped to Search Guard Classic groups;
create an App Registration for Investigate;
configure Search Guard Classic to validate tokens issued by Active Directory;
configure Investigate to handle user authentication through Active Directory.

App Registration

In order to create a new App Registration, login to the Azure portal and switch to the Active Directory blade; then, select App Registrations and click on New registration:

Azure AD App Registrations

Enter an appropriate display name (e.g. Investigate) and select the supported account types, which will be Accounts in this organization directory only for most use cases.

The Redirect URI must be specified as <public Investigate URL>/investigate-access-control/openid/callback; for example, if users are opening Investigate at https://investigate.local, the Redirect URI will be https://investigate.local/investigate-access-control/openid/callback, e.g. :

New app Registration

Take a note of the App Registration parameters, in particular the Application (client) ID; this is the client identifier that Investigate will use to retrieve the information about a logged in user.

App Registration parameters

Then, click on Endpoints and take a note of the OpenID Connect metadata document URL, which is the discovery URL for the App Registration:

App Registration endpoints

In order to put Active Directory groups in the ID token, open the App Registration and click on Token Configuration:

Token configuration

Then click on Add groups claim and add the Group ID to the ID token:

Groups claim settings

Depending on your authorization needs, you can also add Directory Roles or All Groups instead of just groups.

Finally, you’ll need to create a set of credentials to grant Investigate the permission to initiate the authentication flow; in order to create the credentials, click on Certificates and secrets and then on New client secret.

Client secret

Take a note of the secret as it won’t be displayed anymore and make sure to set a reminder when the secret expires as authentication will stop working.

Search Guard Classic configuration

Group mapping

Active Directory groups can be mapped to Search Guard Classic roles by altering the sg_roles_mapping.yml file in the Search Guard Classic configuration directory; for example, if you want to map the investigate_admin role to an Active Directory group with the ID c0a71abc-3a4f-4302-ae1f-1d52620f6123, the Active Directory group id needs to be added to the backendroles list:

investigate_admin:
  users:
  - sirenadmin
  backendroles:
  - 'c0a71465-3a4f-4302-ae1f-1d52620f6abc'

The identifiers of Active Directory groups can be retrieved from the Active Directory blade on Azure by clicking on Groups:

Azure AD groups

OpenID Connect settings

The following sample sg_config.yml configures Search Guard Classic to work with Azure Active Directory claims ; make sure to set the correct openid_connect_url by pasting the value of the OpenID connect metadata document endpoint.

searchguard:
  dynamic:
    respect_request_indices_options: true
    kibana:
      do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      # The standard authentication domain using the internal user database
      basic_internal_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: internal
      # The additional OpenID connect authentication domain
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            # The name of the ID token claim that contains the unique user identifier
            subject_key: sub
            # The name of the ID token claim that contains the user's Active Directory groups;
            # these will be available in mapping rules as `backendroles`.
            roles_key: groups
            # The discovery URL taken from the App Registration's "OpenID Connect metadata document" endpoint.
            openid_connect_url: "https://login.microsoftonline.com/abcd/v2.0/.well-known/openid-configuration"
            enable_ssl: true
            verify_hostnames: true
        # User information is provided in tokens so there's no need for any
        # authentication backend.
        authentication_backend:
          type: noop

Once you have modified sg_config.yml and sg_roles_mapping.yml, they will need to be uploaded to the cluster using sgadmin.

Siren Investigate configuration

To configure Siren Investigate to work with Azure Active Directory, you will need to set the following variables under the investigate_access_control section:

public_url: the URL used by end users to open Investigate.
backends.searchguard.authenticator: set to openid.
backends.searchguard.openid.discovery_url: set to the OpenID Connect metadata document URL from the App Registration endpoints in Azure.
backends.searchguard.openid.client_id: set to the Application (client) ID from the App Registration overview.
backends.searchguard.openid.client_secret: set to the client secret generated in the App Registration’s "Certificates & Secrets" section.
session.backend: this can be left to false in most scenarios; if you’re experiencing issues initiating a session because the ID token contains a very large number of roles, it is advised to set it to true.

Example configuration:

investigate_access_control:
  enabled: true
  public_url: "https://investigate.local/investigate"
  acl:
    enabled: true
  admin_role: investigate_admin
  session:
    backend: false
  cookie:
    name: 'kac'
    password: '12345678123456781234567812345678'
    secure: true
  backend: searchguard
  backends:
    searchguard:
      authenticator: openid
      openid:
        discovery_url: "https://login.microsoftonline.com/abcd/v2.0/.well-known/openid-configuration"
        client_id: "123-abc-123"
        client_secret: "secret"

Restart Investigate after applying the changes to the investigate_access_control section.

If the configuration is correct, when opening Investigate you should be redirected to a login page on microsoft.com:

Client secret

After approval, the user should be logged into Investigate and should have the expected Search Guard Classic roles (displayed as Elasticsearch roles) - The GUIDs of the Active Directory groups should be listed in the list of DLS roles.

User information

Elasticsearch logging configuration

With both basic authentication and OpenID Connect enabled, several versions of Search Guard Classic will log many warnings such as:

[...][WARN ][c.f.o.s.h.HTTPBasicAuthenticator] [iUzXe2s] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'

To silence those warnings, you can add the following lines to the log4j2.properties file in the Elasticsearch configuration directory on each node:

logger.basicauth.name = com.floragunn.searchguard.http.HTTPBasicAuthenticator
logger.basicauth.level = ERROR

Troubleshooting

If the OpenID Connect discovery URL is incorrect, Investigate will display a status page with a discovery error. If that happens, head to the Endpoints page of the App Registration blade and double check it has been pasted correctly, then restart Investigate.

If the OpenID client_id is incorrect, you will get an "Application not found" error from microsoft.com; if the client ID is correct but the secret is not, Investigate won’t be able to initiate the authentication flow and will display a generic 401 error instead of redirecting to microsoft.com.

If you get an authorization error after approving the login request on microsoft.com, first check that the value of openid_connect_url in the Search Guard Classic configuration is correct by looking at Elasticsearch logs; if the URL is incorrect, you will see a KeySetProvider threw error message and details about the request failure.

If there are no errors related to the OpenID Connect discovery URL, a common error is that the subject_key option is set to a claim name that is not present in the ID token issues by Active Directory; when that happens, you will see a warning like the following:

...[WARN ][.AbstractHTTPJwtAuthenticator] [iUzXe2s] Failed to get subject from JWT claims, check if subject_key '<value of subject_key>' is correct.
...[ERROR][.AbstractHTTPJwtAuthenticator] [iUzXe2s] No subject found in JWT token

If the user is authenticated but it does not have the expected roles, make sure that:

roles_key is set to groups;
sg_roles_mapping.yml contains rules mapping the group identifiers to Search Guard Classic roles;
the groups claim is added to the ID token by the App Registration token configuration in the Azure Portal.

ID token inspection

If you need to inspect the ID token, you can execute the authorization code flow using your browser and Postman.

First, download and install Postman from https://www.postman.com/downloads/ .

Then, open the Active Directory developer documentation at https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow#request-an-access-token and click on Run in Postman to get the Azure AD v2.0 Protocols collection.

Postman

Expand OAuth 2.0 Authorization Code Flow and select Authorize Request, then set Query params as follows:

client_id: the App Registration client ID.
response_type: code.
response_mode: query.
scope: openid.
redirect_uri: the Investigate redirect URI that was set in the App Registration (e.g. 'https://investigate.local/investigate-access-control/openid/callback').
state: a random number.

Replace common in the Postman URL with with the Tenant ID that can be retrieved from the App Registration homepage and save the request:

App Registration parameters

Then, stop Investigate and open the URL in your browser and log in; after consent, copy the value of the code parameter in the redirect URL querystring:

Authorization code

Switch back to Postman and select the Token Request - Auth Code request, then set Body parameters as follows:

client_id: the App Registration client ID.
scope: openid.
redirect_uri: the Investigate redirect URI that was set in the App Registration (e.g. 'https://investigate.local/investigate-access-control/openid/callback').
grant_type: authorization_code.
client_secret: the client secret set in the Investigate configuration.
code: the code copied from the querystring in the previous step.

Replace common in the Postman URL with with the Tenant ID that can be retrieved from the App Registration homepage and save the request.

Click on Send to send the request; you should get a JSON encoded response containing the access token and ID token.

You can now paste the encoded token into a tool like https://jwt.ms to inspect its claims.

An authorization code can be used only once, so if you want to repeat the request you’ll have to request a new authorization code by repeating the steps above.

Next steps

Configuring ACL in Siren Investigate