Release Notes

12.0.2

Security fixes

  • Fixed an issue that prevented the access control index from being migrated when upgrading from version 10.5.0 to version 12.0.0 or 12.0.1.

  • Upgraded Node.js to version 14.18.3. For the full list of fixes, see the 14.8.3 changelog.

  • Fixed the Monaco editor configuration to avoid loading parts of the codebase from the jsdelivr CDN.

Bug fixes

Migrations

  • Fixed an issue that prevented upgrading from version 11.0.0 to version 12.0.1.

  • Fixed an issue that prevented upgrading an empty Siren Investigate installation with web services enabled.

  • Fixed an issue that caused filters saved to shared dashboard URLs to not be restored correctly.

Data Model

  • Made the ordering of fields in the target table consistent with the ordering of fields in the source table.

  • Fixed an issue that prevented the save button from being enabled after changing the primary time field or enabling revisions on an entity table.

  • Fixed an issue that caused an error to be displayed when generating a dashboard immediately after creating an entity table.

  • Fixed an issue that prevented field filtering from working in the Search tab field list.

Dashboard

  • Fixed an issue that caused the count of documents to be incorrect in the presence of a filter created from a visualization linked to a search different than the one associated to the dashboard.

  • Fixed an issue that prevented the number of surrounding documents from being displayed.

  • Fixed an issue that prevented a confirmation dialog from being shown when closing a record editing view with unsaved changes.

Visualizations

  • Fixed an issue that prevented horizontal scrollbars from being displayed on the record table.

  • Fixed the Record Table visualization to allow selecting nested fields when exporting data.

Jira Plugin

  • Fixed an issue that prevented results from being displayed when clearing the filter bar.

Internationalization

  • Fixed i18n scripts to be runnable in release builds.

Setup

  • Added the permissions to manage pipelines to the investigate_system role in the Elasticsearch Security initialization script.

  • Added permissions to manage ingest pipelines and templates plus permission to write to data indices to the federate_system role in the Elasticsearch Security initialization script.

Management

  • Fixed an issue that prevented the title of scripts and templates from being displayed in the breadcrumbs of the Management section.

Improvements

Dashboard

  • Improved the Dashboard 360 loading time by optimising the computation of search requests.

  • Dashboard 360 data model pop-ups are now disabled by default to improve performance. To re-enable them you can set the Advanced Setting siren:dashboards360:showDashboardDataModelPopup to true.

  • Added support for ip fields in quick dashboard generation.

Graph Browser

  • Context menus can be closed by pressing the ESC key.

Global Search

  • When entity tables with an external revision index are configured to be searchable, search requests will be executed using the dfs_query_then_fetch mode to ensure that the ranking of edited documents is correct.

Data Model

  • When adding data only writeable tables are displayed.

Dependencies

  • Upgraded React to version 16.14.0.

12.0.1

Bug fixes

Migrations

  • Fixed an issue that caused a migration to fail when the web_service_manager plugin was enabled.

12.0.0

New features

Global search interface

A new Global Search interface is available, which provides instant access to all of the data within Siren Platform’s reach in a single, unified interface. After you find a record of interest, you can view it in more detail. Learn more.

Adding and editing data with revision tracking

You can add or edit records in an existing entity table by using the Record View, which is available in the Global Search interface, in the Graph Browser, in the Record Table visualization, and in the Data model app. Learn more.

Adding links in the Graph Browser

A new contextual function is available in the Graph Browser, which allows you to add or edit links between nodes in the graph.

This allows you to curate the graph for investigations where you are aware of connections between entities that are not displayed. Learn more.

You can also limit the visual clutter in a graph by transforming interconnecting nodes into links (edges) with a newly-developed lens. Learn more.

Importing data in the Data model app

The Data model app has a new interface that provides extensive data import capabilities. It is now a comprehensive location for working with your data; from importing to editing, to adding relations between entities.

You can import data either from a spreadsheet or from a datasource and apply a transform operation by using simple drag and drop functions. The transform can be saved and reused later on new data files that have a similar structure. Learn more.

New features in the Record View

Now, when you expand a record to view its details, the Record View includes the ability to view linked records and to filter records to a dashboard. Learn more.

New linking options for visualizations

If you need to switch the data set that is linked to a visualization, you can select a different entity table or search on the configuration screen.

You can also allow a visualization to be reused on multiple dashboards. Learn more.

Taxonomy browser

In use cases where precision drill-downs are required - for example, when examining technical terms for patents, technical documentation, or scientific literature - Siren Investigate now provides a taxonomy browser.

Along with improved taxonomy annotation capabilities in our built-in natural language processing (NLP), analysts can explore advanced technical and domain-specific datasets and can choose taxonomical categories within a tree visualization.

To add a taxonomy browser to a dashboard, configure and add a Controls visualization with an options tree. Learn more.

Updated terminology in Siren Investigate

'Searches' or 'index pattern searches' are now referred to as entity tables. And entity tables can have subordinates, now called searches. For more information, see the data model documentation and the Glossary.

Compatibility with Avatica for datasource configuration

You can import data into entity tables from Avatica instances that are configured in Siren Federate. Avatica is now the only JDBC framework option that is supported for connecting to datasources and replaces all previous options. Learn more.

Security fixes

Incorrect access control in the the Dev Tools REST API for logged-in users (CVE-2021-43263)

In Siren Investigate versions previous to 11.1.7, the Dev Tools proxy REST API was not checking the permission to view the Dev Tools application. This allowed logged-in users to send requests even without access to the Dev Tools section of the UI. The requests were executed with user privileges. This issue has been fixed in Siren Investigate 11.1.7 and 12.0.0.

Example sic_user and investigate_user roles permissions

The example security configuration for the sic_user and investigate_user roles has been modified to allow read-only access to the siren-import- prefix instead of siren-. This measure prevents access to siren-audit- indices when auditing is enabled and when the indices are configured to store data on the same cluster. It is recommended that an administrator alters the sic_user and investigate_user roles configuration to grant access only to the siren-import- prefix, even if you are not using the auditing feature.

Node.js upgrade

Upgraded Node.js to version 14.18.1. For the full list of fixes, see the 14.8.1 changelog.

Breaking Changes

Internationalization

  • i18n keys prefixed with sirenPlugins.accessControl now must be prefixed with kbn.accessControl instead.

  • i18n keys prefixed with sirenPlugins.ingest now must be prefixed with kbn.ingest instead.

Data import

  • The support for import templates that created Siren Investigate scoped indices has been removed. Entity tables that point to existing scoped indices will work as before but, in order to add new data, you will need to grant write permissions to these indices in the Elasticsearch security configuration.

Graph Browser

  • The signature of the expandByRelation sirenAPI function is now updated. The node ids and relation ids must be passed as parameters of the options object as expandByRelation({ nodeIds, relationIds}).

  • The index-pattern saved object type is now deprecated. If you created lenses to process relations obtained through the f.getKibiRelations method and you need to retrieve the Elasticsearch index pattern that is associated with a search, you can call relation.range._objects.savedSearch.getIndexPattern().

Visualizations

  • It is no longer possible to create new Coordinate Map visualizations. Existing visualizations remain functional, however, it is recommended that you recreate them as Enhanced Coordinate Map visualizations to avoid technical issues.

Deprecations

Search Guard admin certificate

Support for using a local TLS admin client certificate to send requests from the Search Guard configuration UI has been deprecated and will be removed in Siren Investigate version 13.0.

To prepare for this change and also to improve the security of your existing installation, you will first need to declare the roles that are allowed to access the Search Guard management REST API in the elasticsearch.yml file of all the nodes in your cluster. For example, add the following line:

searchguard.restapi.roles_enabled: ["investigate_admin"]

Next, remove the following options from the investigate.yml configuration file and the files that are defined by them from the computer where Siren Investigate is installed:

  • investigate_access_control.backends.searchguard.admin.ssl.cert

  • investigate_access_control.backends.searchguard.admin.ssl.key

  • investigate_access_control.backends.searchguard.admin.ssl.keyPassphrase

To validate the change, restart both the Elasticsearch cluster and Siren Investigate, go to the Authentication tab of the Access control app, and ensure that the list of users or roles is displayed.

Index pattern service deprecation

The indexPatterns service is now deprecated and should not be used. Instead of fetching indexPattern saved objects, the savedSearches service should be used to fetch savedSearch saved objects.

Each 'savedSearch' object contains all of the methods that used to be available in an indexPattern object. To learn more about all of the available methods, see the following file: src/core_plugins/kibana/public/discover/saved_searches/_saved_search.js.

The deprecated service will be removed in a future release. For support with migrating your plugin, go to the Siren Support Portal.

REST API

  • The /investigate-access-control/api/v1/acl/object-permissions/objects route has been deprecated and will be be removed in the next major release.

  • Experimental support for Kable and Timelion in Siren Alert has been deprecated and will be removed in a future release.

Custom watchers

  • The second argument for custom watcher conditions, which contains the raw mappings of the Elasticsearch index that the current dashboard is pointing to has been deprecated and will be removed in a future release.

Script types: contextual and onGraphUpdate

  • The contextual and onGraphUpdate script types are deprecated and are no longer supported as of the next major release. All of the features that were provided by the built-in contextual and onGraphUpdate scripts are now part of the core Graph Browser codebase. You can now write custom scripts by using the facilities that are provided by the Scripting API.

Improvements

Backup and restore

  • Backups of system indices taken in version 12.0.0 and later will be restored to the indices specified in investigate.yml to simplify the creation of new Siren Investigate instances. Backups taken in previous releases will be still restored to the index they were created from.

Data Model

  • Reduced the number of buckets in aggregations used by the dashboard generation wizard from 100 to 20 to avoid performance issues on large indices.

Visualizations

  • Removed the ability to sort columns on fields of type text that do not have a keyword subfield in Records Table visualizations.

  • Added the ability to sort columns on fields using a tag formatter.

Graph Browser

  • Nodes are shown as images automatically for entity tables that have a Default image set in the Data model app.

  • Added missing tooltips to toolbar buttons.

  • Fixed some keyboard shortcuts that were not working properly (spacebar and numeric pad keys).

  • Improved the sizing algorithm for nodes in Map mode.

  • Improved the rendering performance of Time mode.

Scripting

  • The scripts editor is now based on the Monaco open source library.

  • Added axios to the list of libraries that can be enabled through the siren_scripting.librariesWhitelist configuration setting.

Saved objects

  • When you import saved objects from a previous version of Siren Investigate, a confirmation dialog is displayed that reminds the user to execute migrations.

  • Importing objects from future Siren Investigate releases is now forbidden.

  • It is now forbidden to delete index pattern saved objects from the Saved Objects in the Management app.

  • The breadcrumbs now show the saved object title instead of its UUID.

Bug fixes

Dashboard

  • Fixed an issue that would cause an extra count query in certain cases.

  • Fixed an issue that prevented the Time filter widget from appearing automatically on dashboards after changing the underlying entity table to be time-based.

Enhanced Coordinate Map

  • Fixed an issue that prevented the removal of layers from the map when zooming to a location without points of interest.

Graph Browser

  • Fixed an issue that caused time properties that were set by a lens not to be restored when deactivating the lens.

Data Model

  • Fixed an issue that could cause the deletion of an entity table when canceling the automatic generation of a dashboard.

Jira integration

  • Fixed the pixel ratio for images attached to issues on hi-dpi screens.