Auditing user sessions

If you want to perform internal audits, you can enable the session audit feature. This feature allows you to log the actions that are performed by the logged-in user.

Configuring session logs

To enable the session audit feature, set the enabled parameter to true in the following code in the investigate.yml file.

  enabled: true                       # The value is false by default.
  temporary_logs_folder: './optimize' # The folder where the system can store the unsaved entries during shutdown or restart. The default value is the './optimize' folder.
  logged_types:                       # The type of data that is logged. The default value is an array that contains all possible types: ui, data, savedObjects, acl, and count.
    - ui
    - search
    - savedObject
    - count
    - dataExport
  log_response_body: false            # When true the system will log the response body for search requests and savedObject requests
  log_header_names: []                # Array of header names to be logged in addition to all headers with "siren-" prefix which are always logged when audit is enabled
  log_remote_ip_from_header_name: 'my-header' # By default Investigate will check for x-forwarded-for header and extract the first ip address from it
                                              # Sometimes a non standard header is used to store the client ip address
                                              # In such case this property allows to specify such header name
  # If you do not define the outputs, the system uses the main Elasticsearch cluster to store audit entries.
  # It is possible to configure more than one remote Elasticsearch output.
    - elasticsearch
        manage_template: true          # default true
        queue_size: 1000               # default 1000
        flush_interval: 10000          # default 10000 ms
        flush_size: 50                 # default 50
        index_prefix: 'siren-audit-'   # default siren-audit-
        index_interval: daily|monthly  # default monthly
        number_of_shards: 1            # default 1
        number_of_replicas: 1          # default 1
        url: https://host:port
        username: username
        password: password
          verificationMode: none | certificate | full  # default full
          certificateAuthorities: [ "/path/to/your/CA.pem" ]
          certificate: /path/to/your/client.crt
          key: /path/to/your/client.key
          keyPassphrase: changeme

# When set, it is written to the instanceName field of audit entries
# Useful when multiple Investigate instances are configured to log audit entries to the same Elasticsearch cluster
investigate_core.instance_name: "my-instance-01"

Session log indices

After the feature is enabled, the system automatically creates an index that stores the audit entries. If you use the default value of elasticsearch as the output type, the system creates the index by using the output configuration credentials. If no output is defined, the system creates the index on the main Elasticsearch cluster by using the Siren Investigate System credentials.

Make sure that either the remote cluster user or the Siren Investigate Administrator user has the following security permissions:

  • Can store an index template.

  • Can create indices with a name starting with the configured prefix, for example, the default prefix is siren-audit-.

  • Can write to those indices.

Session log entries

The system can log the following types of session data:

  • ui: Actions that are performed by the user as they are interacting with the application. For example, a log entry is created when a user opens a dashboard.

  • savedObject: Access to system configuration objects. For example, a log entry is created when a user requests a configuration object for particular dashboard.

  • search: All data-related search queries. For example, a log entry is created when a visualization on a dashboard is fetching the data to render.

  • count: All queries that obtain the document count. For example, a log entry is created when a query is used to fetch the count of documents visible on a dashboard.

  • dataExport: A log entry is created each time a user attempts to export data as JSON or CSV.

When the log_response_body option is set to true, you must use a different instance of Siren Investigate to view the session audit logs - one that has auditing disabled. If the same instance is used to view the logs, the log entries will quickly grow in size, causing errors. This is due to recursion; a request logs a response, which when viewed again, again logs a response.

It is not currently possible to monitor the status of the audit index. In the event that the output remote cluster is down, the system can accumulate the log entries in a back-end, in-memory queue. If the queue reaches its maximum size, the system starts to lose the oldest audit entries. To avoid data loss, system administrators must actively monitor the health of remote audit clusters to keep them online.

For all 10.5.x versions of Investigate the Elasticsearch used as an output for session audit logs must be >= 6.8.0