Siren Platform User Guide

Siren Investigate 10.0.0

Siren Investigate Changes

Added:

  • Added Elasticsearch 5.6.9 compatibility.
  • Added a JDBC datasource browser that enables the user to browse the datasource that is used when creating a virtual index and to select which table to import.
  • Now the system offers to automatically add a saved search when creating an index pattern.
  • After index creation, the user is now taken to the new index’s edit page for modification, if needed.
  • EID buttons now reflect changes to counts in the data, for example after applying a filter.
  • Added a user confirmation to the CLI upgrade procedure to check if the user has backed up their .siren index.
  • Investigate now handles empty index patterns more gracefully.
  • The relational graph in the Indexes and Relations section is moved to a tab.
  • Total Duration time of a request is now displayed on the Spy Panel.
  • Added config file migration for investigate.ymls to enable migration between post-10 versions.
  • Added migrations for custom configuration .ymls or .ymls in custom folders.

Fixed:

  • Fixed bug in the Relational Navigator when creating a dashboard without a saved search.
  • Fixed bug where the Relational Navigator would show an EID button, even though there was no destination dashboard.
  • A number of fixes to the upgrade backup process:

    • Now the backup files are backed up to the /data folder
    • Enable the user to specify a custom backup folder
    • Changed backup folder names to use ISO datetimes for timestamp
    • The index is removed and restored from scratch if there is a problem to prevent extra objects from the new index remaining.
  • Fixed missing docs link in time filter creator.
  • Fixed visibility toggle on the Relational Navigator - now buttons are hidden when configured in the visualization.
  • Autoselect now does not discard multifields if their parent is unselectable, for example it is not aggregatable
  • Fixed Dashboard sidebar drag and drop UI to make it clearer the dashboard is being dragged when grabbed with the cursor.
  • Fixed explanation when a filter was negated - now says NOT ….
  • Fixed bug in the Relational Navigator where the buttons were not shown on an index pattern with no relations.
  • Fixed a bug with filters being merged with the state unnecessarily causing issues on dashboard reload.
  • Now deleting an index pattern in Indexes and Relations updates the list so the deleted index pattern is removed.
  • Fixed bug in rendering the TagCloud visualization that would cause a browser crash on tag cloud load.
  • Sorting is now possible again in the Enhanced Table visualization.
  • Fixed filter selection icons showing in each column of a row when moving the mouse pointer over a cell.
  • Fixed a bug where multiple filters from individual relational buttons could be added to the elasticsearch request.
  • Now the names of the datasources can be edited after they have been saved.
  • Now returning more explanation if your query fails because of an Out Of Memory exception.
  • A wildcard query on a dashboard no longer shows a filter icon on the dashboard sidebar.
  • Fixed a bug in relational buttons that would remove parts of the state if a request from the button was null.
  • Completely refactored how automatically generated buttons are rendered to handle the number of requests sent on dashboard navigation.
  • Fixed 'Hide Borders' function. It now hides the borders.
  • Text filter to search relations and edit relational buttons now responds to text input.
  • Now the date is reset when the user cancels an edit in a saved dashboard.
  • Relations with no destination other than the EID are not listed in the automatic relational buttons.
  • Fixed a crash when filtering visualisations.
  • Added support for siren:timePrecision back in.
  • URL shortener in Dashboard Share panel now generating shortened URLs correctly.
  • Fixed intermittent error where dashboard ID was not passed correctly to relational buttons.
  • Enable creation of index pattern directly from create virtual index page without manually editing index pattern name.
  • Fixed bug in saving dashboard in Saved Objects after making no changes.
  • Spy panel now only listed permitted modes.
  • Users trying to access dashboards or index patterns without ACL permissions are shown more graceful errors.
  • The dashboard sidebar and relational buttons now show warning symbols when attempting to get counts from un-authorized dashboards.
  • Fixed bug when 500 error returned if attempting to edit an index pattern without permissions.
  • Config file validation check now runs when the upgrade CLI command is run.
  • Config file migration now accepts custom config files/folders.
  • The timefilter dashboard sync panel is now shown even if the user is denied access to the dashboard by ACL.
  • Fixed a crash when clicking colorpicker in Timeseries visualization configuration.
  • Now returning an error if there is no config file in the config folder.
  • Newly created relation labels are available for selection in other relations without a save.
  • Auto dashboard generation

    • Visualisations created with Generate Dashboard were not associated to the saved search, now they are.
    • Storing the time in the dashboard now causes the generated visualizations to fit the target time interval.
    • Fixes issues with sidebar dashboard counts after a generation, like neverending spinners.
    • Added a report for Generate, that enables users to change visualization titles.
    • Both Autoselect and Generate reports enable sorting by column and selection of output items.
    • Improved filtering of common undesired distributions in Autoselect.
    • More descriptive visualization names in Generate.
  • Minor UI fixes:

    • Dashboard sidebar click and drag functionality improved.
    • Siren Investigate logo quality improved.
    • Sidebar scrollbar color was changed to match the theme.
    • Position of Home tooltip on logo was fixed.

Changed:

  • All the icons have been changed to FontAwesome 5 Pro versions.
  • Impala has been added to the list of available JDBC datasources.
  • The segmented request logic for discover page to prevent the doc table in Discover trying to request the same data again.
  • Merged changes from Kibana 5.6.8 and Kibana 5.6.9.
  • Changed to consistently use match_all: {} queries instead of query_string: { query: '*' }.
  • Table visualizations header styling was improved to reduce white space between columns.
  • Added a note to inform about the upcoming deprecation of the Relational Filter visualization.
  • EIDs are now prioritized in automatic dashboard field selections.
  • Added selection per row for filter creation in the Enhanced Table visualization.
  • Improved the dashboard highlight colour.
  • Now the first index pattern that is created is automatically set as the default index pattern.
  • Now the upgrade command backs up the configuration index by default.
  • Removed some redundant advanced settings (for example siren:zoom).
  • JDBC datasources have been removed from the management/saved_objects page.

Known issues:

  • Unzipping siren platform on a Windows OS may result in some errors as the file path is too long. For this reason it is recommended to unzip using a package like 7zip which will unzip normally and ignore these errors or to unzip in a top level folder with a single character as the folder name, for example "C:\s"

Plugins

Note

Siren plugins can now be found in the siren_plugins/ folder but any third-party plugins should still be installed into the plugins/ folder.

Graph browser

Improved:

  • Various performance improvements:

    • Improved performance by optimizing the serialization of sessions.
    • now handles the addition of several entities quickly.
    • Reduced the request payload to improve response times.
    • Better handling of more than 1024 nodes.
    • Selection algorithm was improved to help data selection changes.
    • Now batch sending requests on expansion - leading to increased responsiveness.
  • Better consistency in link directions.
  • Optimized edge-count strategy, reducing time spent on expand actions by half, in some cases.
  • Rewrote the logic to compute counts for nodes. Big speed up, no more missing relations.
  • Now supports nodes with millions of relations.

Fixed:

  • Fixed bug where EIDs would not show on expansion.
  • Stopped unnecessary HTTP calls if the license was invalid/missing.
  • Fixed a bug where canceling a lazy loading in the graph caused the browser to hang.

Added:

  • Graph browser functions have moved into a sidebar that enables listing, display and manipulation of the data and filters in tabular format.
  • Select edge script now works when relation count = 1.
  • Graph browser now handles nested index patterns and multiple entities matching an index pattern.
  • A button is added to show inverse relations on the graph.
  • Changes in the graph now persist when navigating to other tabs.
  • Arrows are added to the relations for Entities unless the labels for both relationships are the same.
  • Added Graph Browser sidebar Lenses:

    • Now you can navigate through your data on the Graph Browser, select data, apply functions and transformations to the data.
    • The Graph Browser ships with scripts to transform your data’s size, colour, and so on based on a field.
  • Added the ability to manually add EIDs to a graph.
  • Added a check box to show nodes on the graph without time fields when using the timeline.
  • Added exclude configuration to fields to enable the user to remove extraneous fields from the graph.
  • When expanding a large node, now the user can choose to retrieve a subselection (the amount retrieved is configurable).

Known issues:

  • After migration from version 5.x.x graph icons, colors, and custom labels will be lost. User have to reset these values for each index pattern by going to Management → Indexes and Relations then select an index-pattern and set the values under the "generic" tab where: Custom Labels - Instance Label → Scripted Label Color - Color Icon - Icon This is partially caused by an upgrade of FontAwesome library from v4 to v5 which comes with many more awesome icons.
Access control

Fixed:

  • Fixed indentation in the default Role template.
  • Now enables tabs in the Role templates.
  • Fixed a bug when deleting a duplicated rule has no effect.
  • Now an error shows on the login page if there is no connection to Elasticsearch.

Added:

  • Added admin.ssl.keyPassphrase option to searchguard ssl options.
Multichart

Added:

  • Added a button to enable the renaming of configurations.
  • Added a warning when you click "Get Default Configuration" that your current configurations will be destroyed.
  • Added a warning when a field is in a configuration but not in the associated index pattern.

Fixed:

  • Fixed a bug where a query with no data in the field would return an error.
  • Fixed bug in Next arrow where it would return to the beginning of the list each time.
Gremlin Server

Fixed:

  • Siren Gremlin Server now checks that an index mapping exists before trying to fetch the mapping. This prevents a non-blocking error report on startup with no index-pattern.
  • The Siren Gremlin Server will now shutdown if the connection to Elasticsearch is not available.
Scatterplot
  • The text in the legend no longer overflows the legend box.
  • The color picker is now back beside the hex colour input box.
License
  • The check for a valid license is now cached for an hour, leading to improved performance when navigating between routes.
  • If a user without permissions attempts to upload a license then an error is shown.

Search results

    No results found