Release Notes

11.1.10

Security fixes

  • Upgraded Node.js from version 14.18.1 to version 14.18.3. For the full list of fixes, see the 14.8.3 changelog.

Bug fixes

Data Model

  • Fixed an error in the automatic dashboard generation procedure that occurred in the presence of fields of type ip.

Dashboard

  • Fixed an issue that caused the count of documents in a dashboard to be wrong when a filter associated to a different search was enabled.

  • Fixed an issue that caused the funnel icon to appear in the sidebar when no filters were applied.

  • Prevented scrolling the dashboard sidebar from changing the scroll position of the active dashboard.

  • Fixed a revision filter being incorrectly injected when cloning a 360 Dashboard.

  • Fixed an issue that prevented a newly created dashboard from appearing in the sidebar after being created in environments with slow network performance.

Visualizations

  • Fixed an issue that caused the count of documents in a nested slice to be incorrect when enabling the "Other" bucket.

  • Fixed the aggregation legend appearing in Enhanced Tilemap visualizations after disabling the aggregation layer.

  • Fixed a fatal error when clicking twice on the Export button of Record Table visualizations in environments with slow network performance.

  • Fixed the labels on the X axis appearing as undefined when using a Range aggregation.

  • Fixed an error when exporting a Record Table visualization in a dataspace where the advanced setting metaFields was set to null.

  • Fixed an error when exporting a Record Table visualization that was sorted by a missing field.

  • Fixed an incorrect value being displayed in the legend when setting a custom label for an aggregation.

Jira integration

  • Fixed the reporting of authorization errors in the issue selector dialog.

Miscellaneous

  • Fixed an issue that could cause Siren Investigate to get stuck during the startup process if the Elasticsearch cluster crashed.

11.1.9

Bug fixes

Migrations

  • Fixed an issue that prevented saved objects created by the Web Services plugin inside dataspaces from being upgraded.

11.1.8

Breaking changes in version 11.1.8

  • Dashboard 360 data model pop-ups are now disabled by default to improve performance. To re-enable them you can set the Advanced Setting siren:dashboards360:showDashboardDataModelPopup to true.

For the breaking changes that are relevant to Siren Investigate version 11.1.0, see this section.

Security fixes

  • Upgraded the axios dependency to version 0.24.0 to address CVE-2021-3749.

Bug fixes

Migrations

  • Fixed an issue that prevented the migration of Saved Object ACL rules when upgrading from Siren Investigate 10.5.x.

Dashboard

  • Fixed an issue that caused the headers of Record Table visualizations to not align with contents when scrolling horizontally or paging.

  • Fixed an issue that caused an unnecessary delay when adding a new visualization to a dashboard.

  • Improved rendering performance of Record Table visualisations by reducing the number of DOM elements.

  • Improved the Dashboard 360 loading time by optimising the computation of search requests.

Visualizations

  • Fixed an issue that prevented the selection of nested fields when exporting data from Record Table visualizations.

11.1.7

Security fixes

Incorrect access control in the the Dev Tools REST API for logged-in users (CVE-2021-43263)

In Siren Investigate versions previous to 11.1.7, the Dev Tools proxy REST API was not checking the permission to view the Dev Tools application. This allowed logged-in users to send requests even without access to the Dev Tools section of the UI. The requests were executed with user privileges. This issue has been fixed in Siren Investigate 11.1.7.

Example sic_user and investigate_user roles permissions

The example security configuration for the sic_user and investigate_user roles has been modified to allow read-only access to the siren-import- prefix instead of siren-. This measure prevents access to siren-audit- indices when auditing is enabled and when the indices are configured to store data on the same cluster. It is recommended that an administrator alters the sic_user and investigate_user roles configuration to grant access only to the siren-import- prefix, even if you are not using the auditing feature.

Node.js upgrade

Upgraded Node.js from version 14.17.6 to version 14.18.1. For the full list of fixes, see the 14.8.1 changelog.

Bug fixes

Authentication

  • Fixed an issue that prevented the original dataspace from being preserved in the URL after a redirection to Siren Investigate from an OpenID Connect provider.

Dashboard

  • Fixed an issue that caused the layout of the sidebar to break when a user with limited privileges changed the order of the dashboards.

  • Fixed an issue in Dashboard 360 that forced an inner join to be selected when a search on a child node had a saved filter.

  • Fixed an issue in dataspaces that prevented a user with read-write permissions from rearranging items in the sidebar.

  • Record view modals are closed automatically when a new dashboard search is executed.

  • Fixed an issue that caused all visible dashboard counts to be refreshed when a user was viewing a dashboard with the auto-refresh mode enabled. Introduced the timepicker:autoRefreshAllDashboardCount advanced option in the Management app to allow this behavior to be enabled if required.

  • Fixed a regression introduced in Investigate 11.1.5 where a self relation through an EID would produce wrong results when used in Dashboard 360.

Graph Browser

  • Fixed an issue that caused a regression in performance when manipulating graphs with thousands of nodes.

  • Upgraded dependencies to address an issue that could cause the graph to not be rendered or crash on Chrome version 93 or higher.

  • Fixed an issue that prevented immediately selecting or expanding a node when a tooltip was displayed.

Data import

  • Fixed an issue that prevented uploading CSV files when a custom pipeline was specified in the import configuration.

Visualizations

  • Fixed an issue in the Analytic Table visualization that caused incorrect counts when configuring multiple buckets for both rows and columns.

  • Fixed an issue in the Timeline visualization that caused incorrect fields to be displayed after selecting a search in the configuration panel.

Scripting

  • Fixed an issue in the dashboard.getFilters method that caused outdated filters to be returned.

Breaking changes

For the breaking changes that are relevant to earlier versions of Siren Investigate, see the sections for version 11.1.1 and version 11.1.0.

11.1.6

Security fixes

Bug fixes

Data Model

  • Fixed an issue that caused an error when fetching index fields when support for large numbers was enabled.

Dashboard

  • Fixed an issue that caused extra keyword fields to appear in exported CSV files.

  • Fixed a crash occurring when clicking on the zoom icon of the data model editor in 360 dashboards.

Graph browser

  • Applied a critical licensing fix to third-party dependencies.

  • Fixed an issue that caused time fields with a formatter to appear as numerical timestamps in tooltips.

Records table visualization

  • Fixed an issue that prevented filtering buttons from appearing on newly added columns.

Siren Alert

  • Fixed an issue that caused the first custom action defined after creating a new watcher to not be saved correctly.

Improvements

  • The auditing engine can be configured to log changes to saved objects performed by users. For more information, see Auditing user sessions.

  • When exporting data as CSV or JSON from a Record Table visualization, the fields displayed in the table will be selected automatically.

11.1.5

Security fixes

  • Upgraded Node.js from version 14.17.3 to version 14.17.5. For the full list of fixes, see the 14.17.5 changelog.

Bug fixes

Migrations

  • Optimized migrations to support large Siren Investigate system indices.

  • Fixed an issue that caused a migration of the data model graph to be requested after Siren Investigate was restarted.

Visualize

  • Fixed an issue that prevented the Filter button from appearing in the visualization editor.

  • Fixed an issue that caused the Visualize button to open the visualization editor in the wrong dataspace.

Data model

  • It is now possible to clear the default label and the default image of saved searches.

Graph Browser

  • Fixed an issue that prevented the scrollbar on the toolbar from being displayed in Mozilla Firefox.

  • Fixed an issue that prevented relationships from being shown in the sidebar for nodes that were added from a child search.

  • Fixed an issue with the Node Image lens that prevented images from being retrieved correctly when the server.basePath option was set in Siren Investigate.

  • Fixed an issue that prevented the time bar from being updated after removing nodes from a graph.

Controls visualization

  • Fixed an issue that prevented the visualization widgets from being correctly cleared when clicking on the Clear button.

  • Fixed an issue that caused the filters created by the visualization from being correctly removed.

Records table visualization

  • Fixed an issue that prevented fields without a value to be displayed when editing an Elasticsearch document or a revision.

  • Fixed an issue that prevented results from being displayed when using pagination during the editing of the visualization.

Miscellaneous

  • Fixed visual glitches in the Relational Navigator visualization.

  • Fixed an issue that caused the dashboard sidebar to lose its scroll position when any dashboard was selected.

11.1.4

Security fixes

Missing TLS verifications (CVE-2021-36794)

In Siren Investigate versions older than 11.1.4, when enabling the cluster feature of the Siren Alert application, TLS verifications are disabled globally in the Siren Investigate main process. As a workaround, it is possible to disable the cluster feature by setting sentinl.settings.cluster.enabled to false in the investigate.yml configuration file. No action is required if the cluster feature was not enabled.

Siren Alert dependency upgrades

Bug fixes

Access Control

  • The SameSite attribute of session cookies is now set explicitly to Lax to prevent redirection loops when using external identity providers.

  • The dataspace shortcode in the URL is now preserved when the user is redirected to the login page.

Data Import

  • Fixed an issue that occurred when importing CSV files that contained emoji characters.

Data Export

  • Fixed an issue that prevented exporting a dashboard displaying data from a remote Elasticsearch cluster.

Visualizations

  • Fixed an issue in the Enhanced Coordinate Map visualization that displayed incorrectly-sized aggregation shapes when changing the zoom level quickly.

  • Fixed an issue in the Controls visualization that caused duplicate values to appear in multiple-selection combo-boxes.

Graph Browser

  • Fixed an issue that prevented tooltips to be displayed when rapidly moving the mouse away from and back onto the same node.

  • Fixed the Select All checkbox in the relation selectors to select only visible relations.

  • Fixed a fatal error that could occur when removing a link representing an aggregated relation from the graph.

  • Fixed an issue that prevented node counts that were added from child searches from being computed.

  • Fixed an issue that prevented multiple new-line characters from being rendered correctly in labels that were created by lens scripts.

Dashboard

  • Fixed an issue that caused entity identifier names to disappear when expanding the Relational Navigator items.

  • Fixed an issue that triggered an extra count request to Elasticsearch when cloning a dashboard.

  • The counts of dashboard inside dashboard groups in the sidebar are only refreshed automatically when the dashboard group is expanded for the first time.

  • Fixed a fatal error in the document JSON viewer that could happen when pressing CTRL+F.

  • Fixed a bug that caused filters created with the filter editor to not work correctly when pinned on 360 dashboards.

Miscellaneous

  • Fixed fatal errors in the Data Import and Visualize applications that occurred when the setting siren:support-large-numbers was enabled.

Improvements

Elasticsearch 7.13 compatibility

Siren Investigate is now compatible with Elasticsearch 7.13.3 clusters that have Siren Federate installed.

User session auditing

  • Added the ability to log the following information:

    • The Siren Investigate instance name.

    • Response bodies.

    • Operations on saved objects.

    • Data export requests.

    • Arbitrary HTTP headers and remote IP addresses.

  • The session audit feature now logs clicks on the application sidebar.

  • Added automatic URI decoding of HTTP header values.

  • Implemented UI entries flushing in batches to prevent errors due to the request size being too large.

Graph Browser

Added layout options to the Expand menu to keep nodes that are already on the graph in the same position when expanding relations.

Miscellaneous

  • Improved the performance of the Graph Browser when switching to map mode with an active node selection.

  • Added the trustAsUrl filter, which can be used in Angular.js templates to process variables through $sce.trustAsResourceUrl.

  • Added the disable_content_type_check setting to the Image proxy component. This setting allows you to disable the Content-Type header verification when you are loading images from servers that are setting it incorrectly.

  • Added the investigate_access_control.session_termination_whitelist.local_storage_keys setting. This setting allows you to preserve some local storage keys on logout for plugin developers.

  • Added the ability to whitelist the jQuery library for use in sirenapi scripts.

Internal changes

  • Removed the obsolete getDocumentFormat method from the Saved Objects API.

11.1.3

Bug fixes

Data Model

  • Fixed an issue that could cause a fatal error when opening the Data Model application on Firefox releases older than 70.

Dashboard

  • Fixed an issue that could prevent the count on Dashboard 360 dashboards from being updated.

Graph Browser

  • Fixed an issue that prevented the Graph Browser from working inside a dataspace with a numerical identifier.

  • Field exclusions defined in the Graph Browser configuration are now applied correctly.

  • Fixed an issue that could cause certain nodes to disappear when switching to the timeline mode.

  • The search selector in the lens configuration panes now shows the search name instead of its UUID.

Miscellaneous

  • Fixed an issue in the Dev Tools application that was causing unnecessary periodic requests after leaving the application.

Improvements

  • The deprecated Gremlin Server component has been removed from Investigate.

11.1.2

Improvements

  • Added the auditing configuration setting wait_for_outputs_initialization which, when enabled, will delay Siren Investigate from starting up until the audit outputs are initialized.

Security fixes

  • Upgraded the lodash dependency to version 4.17.21 to address CVE-2021-23337.

  • Upgraded the apexcharts dependency to version 3.26.2 to address CVE-2021-23327.

  • Upgraded the handlebars dependency to version 4.7.7 to address CVE-2021-23369.

Bug fixes

Authentication

  • Fixed an issue that prevented logging in when using a cluster that is secured by the recent releases of Search Guard for Elasticsearch 7.11 and 7.12.

Dashboard

  • Fixed an issue that prevented Record Table visualizations from being updated when disabling or enabling a live filter.

  • Fixed an issue that prevented any records from being displayed when negating a filter on a Dashboard 360 that contained a relational filter.

  • Fixed an issue in the Dashboard 360 data model where only a subset of relations was displayed for child search targets.

Graph Browser

  • Fixed an issue that limited the number of connections during an expansion when clicking Add all.

  • Fixed an issue that caused date fields to not be displayed correctly in tooltips.

  • Fixed an issue that prevented aggregated relations from working correctly during expansion.

  • Fixed an issue in the Time/Location lens that displayed an unnecessary error message for nodes without a location field.

Miscellaneous

  • Fixed the appearance of saved object titles in the Saved Objects management section.

11.1.1

Breaking changes in version 11.1.1

  • i18n keys that were prefixed with sirenPlugins.accessControl must now be prefixed with kbn.accessControl instead.

  • Support for loading pug templates in plugins has been removed.

For the breaking changes that are relevant to Siren Investigate version 11.1.0, see this section.

Improvements

Support for large numbers

When enabling the siren:support-large-numbers setting in Advanced Settings, numeric fields will now be parsed as BigInt values. This allows the display and creation of filters from values larger than 2^53 - 1, at the cost of a slight performance overhead due to the extra parsing of Elasticsearch responses.

Security fixes

Server-side request forgery in embedded image proxy (CVE-2021-31216)

Siren Investigate versions that were released before 11.1.1 contain a server-side request forgery (SSRF) defect in the built-in image proxy route, which is enabled by default. An attacker with access to the Siren Investigate installation can specify an arbitrary URL in the parameters of the image proxy route and fetch external URLs as the Investigate process on the host.

In version 11.1.1, the image proxy has been disabled by default and can be reactivated if required by following the instructions in this section.

In older versions of Siren Investigate, it is possible to disable the embedded image proxy by adding the following line to the investigate.yml file and restarting the application:

investigate_core:
  image_proxy:
    enabled: false

Node.js

Upgraded Node to version 14.16.1 to address the following CVEs:

Bug fixes

Dashboard

  • Fixed an issue that would prevent automatic scrolling when dragging a visualization across a tall dashboard.

  • Fixed an issue that would cause an error message to appear when switching quickly between a dashboard and the data model application.

  • Fixed an issue that triggered automatic cardinality queries from the relational navigator when the siren:enableAllRelBtnCounts setting was disabled.

  • Fixed an issue that would clear highlight marks in the document view modal after an editing operation.

  • Fixed an issue that could cause incorrect counts on newly cloned dashboards.

  • Fixed the positioning of the sidebar context menu for elements at the bottom of the scrolling viewport.

Data Model

  • Very long index patterns are now truncated with an ellipsis automatically in the Data Model sidebar; the full name can be displayed by either clicking on the search or as a tooltip.

  • Fixed an issue that would cause the relational graph to be incorrectly positioned when resizing the browser window.

  • Fixed an issue that could prevent pages in the Data Model section from being rendered without user interaction.

Logging

  • Fixed a regression that would prevent logs being written when setting logging.dest to a file path.

Graph Browser

  • Fixed the rendering of the Graph Browser in the Visualize application.

  • Fixed an issue that would prevent the user from overwriting a saved graph after making changes to it.

  • Fixed an issue that caused the timeline to reset to the initial period when applying a lens.

  • Fixed an issue in image lenses that would prevent changes to lens parameters from being applied immediately.

  • Fixed an issue that could cause an error when retrieving counts for nodes.

11.1.0

New features

Redesigned Graph Browser toolbar

The Graph Browser toolbar has been completely redesigned to provide a better user experience and lay the foundations for upcoming improvements.

For more information, see Using the toolbar.

User session auditing

The new user session auditing feature allows you to configure Siren Investigate to keep track of user activity by session.

For more information, see Auditing user sessions.

Compatibility with Elastic Cloud and ECE

Siren Investigate can now be used against clusters that are running in Elastic Cloud / ECE and that have a compatible version of the Siren Federate plugin.

Major framework updates

The bundled Node.js package has been upgraded to version 14.15.1 and the hapi package has been upgraded to version 20.0.2 as part of the ongoing effort to modernize the codebase.

Improvements

Miscellaneous

  • The Data Model configuration screen has been moved to a standalone application in the navigation menu.

Scripting

  • When an exception from a script is reported, the error details include the title of the script.

  • Fixed an issue that prevented whitelisting methods on the window.document such as document.getElementById.

Dashboard

  • Fixed an issue that would cause a dashboard to scroll to the first Graph Browser visualization instance upon loading.

  • Disabled the ability to set time filters on root nodes in a Dashboard 360 data model as they are ignored.

  • Fixed an issue that prevented the "Surrounding documents" button from displaying documents.

  • It is now possible to create filters on fields whose name starts with _, with the exclusion of known Elasticsearch metadata fields.

Graph Browser

  • Improved the reporting of errors and timeouts during the execution of shortest path and common communicator algorithms.

  • Added new functions to scripts: defaultExpansionAndParse and getCountsForNodes

Visualizations

  • Fixed a typo in the configuration of the Scatter Plot visualization.

  • Removed animations from the Tag Cloud visualization.

  • Fixed the ellipses in the Relational Navigator buttons.

  • Improved the icon that is displayed in visualizations that do not contain results from Elasticsearch.

  • Fixed an issue that prevented custom sorting from working correctly when using custom templates in the Record Table visualization.

Security

  • It is now possible to configure the TLS version negotiated by Siren Investigate when connecting to Elasticsearch through the elasticsearch.secureProtocol configuration setting. The default value has been set to TLSv1_2_method (TLS 1.2).

  • Clarified in the UI that the permission to view an import configuration implies the ability to use it to import data.

Dependencies

  • Removed the elasticdump dependency for backup commands.

Breaking changes in version 11.1.0

hapi upgrade

The hapi dependency has been upgraded to version 20.0.2 following the end of life of releases older than 19.x. If you developed custom plugins that extend the back-end server, you might need to adjust their code to be compatible with the latest hapi conventions and APIs.

Disabled the jQuery legacy pre-filter by default

In jQuery version 3.5.0 or later, Angular templates with self-closing tags, such as <input/>, are no longer supported. If you have developed any custom plugins that contain templates with self-closing tags, they must be replaced with explicit opening and closing tags, for example, <input></input>, because they will not be rendered otherwise.

If you are not able to fix plugins right away, it is possible to enable a compatibility mode by setting the following options in investigate.yml:

optimize.jqueryLegacyPrefilterEnabled: true
optimize.jqueryMigrateEnabled: true

When these options are enabled, the jQuery Migrate plugin will be enabled and configured to log messages in the console whenever an invalid template is processed.

Graph Browser

  • The parameters of the addDocumentsByQuery sirenAPI function have changed. A saved search id is now used when querying Elasticsearch for entities.

  • Removed the following functions from scripts: executeGremlinQuery and executeGremlinQueryAndParse

Other breaking changes

  • The FieldSelect component has been deprecated and replaced by FieldSelectResponsive. It will be removed in a future release.

  • i18n keys prefixed with sirenPlugins.license now must be prefixed with kbn.license instead.

Security fixes

Bug Fixes

Miscellaneous

  • Fixed an issue that would prevent a resolution dialog from appearing when importing saved objects pointing to missing index patterns.

  • Fixed an issue that could cause a fatal error after deleting a saved search and opening a dashboard that was referring to it.

Data Model

  • Addressed an issue that could prevent the auto-relations wizard from working correctly when selecting Elasticsearch metadata fields in the list of candidates.

  • Fixed an issue that would cause a document details modal to not close automatically when switching between Data Model tabs.

Graph Browser

  • Fixed an issue that could cause numbers in node labels to be forcibly displayed at the end of the label.

Dashboard

  • While a document is being edited, filter buttons are automatically hidden in the document modal.

  • Fixed an issue that could delay or prevent the update of counts in the sidebar when applying a time filter across multiple dashboards.

  • When dragging a dashboard over a dashboard group, the group is now opened automatically.

  • Fixed an issue where a funnel icon was displayed next to a dashboard that did not have additional filters.

  • Fixed an issue where a user with limited access to a subset of dashboards would get redundant error notifications.

  • Resolved an issue that could cause extra count requests when removing filters from a dashboard.

  • Addressed an issue that was causing Elasticsearch field names to be hidden in the document details modal when column aliases were configured.

  • Addressed an issue that prevented the dashboard document count from being updated when removing saved filters.