Dashboards

A Siren Investigate dashboard displays a set of saved visualizations in a customizable grid layout. Each visualization is normally linked to a search. Dashboards can be organized in dashboard groups, shared, and customized.

Since Siren 10.3, the Siren Platform has a significant feature called Dashboard 360. You can now define an internal dashboard data model which enables a dashboard’s visualizations to be based on different tables (searches), and to coherently display only data from connected records at each step. This provides a 360-degree view of a record (or groups of records), showing in a single view what is connected, both directly and indirectly.

The Dashboard Data Model

A Siren Investigate dashboard displays a set of saved visualizations in a customizable grid layout. Dashboards can optionally display a search bar, a filter bar, and (at top right) a ‘time range’.

Investigate provides three types of dashboards:

image

Dashboards that do not represent a search (informational or summary dashboards)

The Dashboard does not represent a search option is used when the dashboard is not primarily about a set of entities, such as companies or investments in the Siren Community demo.

The following screenshot from the demo shows an example of this informational or summary dashboard:

image

Dashboards without a search:

  • Do not display a count (number of records) on top or in the sidebar

  • Cannot be targets or origin of relational navigation (for example, via the Relational Navigator)

With this dashboard type, it is usually better to hide the search/filter bar and the time filter. This can be done from the Options menu in Edit mode.

Dashboards that are about a search

If a dashboard is to represent a specific search (for example, the companies or articles searches), you select the Dashboard is about a search option, and then select the appropriate search from the drop-down menu.

image

This effectively binds the dashboard to the search, which enables the following:

  • The dashboard displays the count on top, as well, the icon associated with the search in question; the count is also displayed on the sidebar.

  • Components like the Relational Navigator start working and can be used on the dashboard (blue links).

  • The content of the dashboard itself can be added to a Graph Browser.

For example, in the following screenshot (from the tutorial demo) the all articles dashboard is set to be about the Articles search. This makes the Relational Navigator component (blue buttons) work and shows the count (646,902) on top and on the sidebar.

image

Dashboard 360: Showing data from multiple connected searches on the same dashboard

Siren 10.3 introduced the Dashboard 360 feature which, thanks to a dashboard-specific data model, enables a single dashboard to contain visualizations that are based on different searches, and to perform coherent filtering across all of them.

In the following example screenshot, the all companies dashboard contains a visualization Companies by Country, which is based on the on the Companies search.

Thanks to the dashboard data model (which will be explained shortly), it is possible to add a visualization (the Investments by Year pie chart) based on the Investments search; clicking on the pie chart then coherently filters the Companies by Country visualization.

image

Note how additional icons appear on the top left of each visualization’s container in Dashboard 360 mode, indicating the searches that are in use.

Creating a data model for Dashboard 360

To get visualizations to be relationally connected (Dashboard 360), we need to create a dashboard data model that defines how the ‘main search’ for the dashboard is to be relationally joined with the other searches that will then power the visualizations.

To configure Dashboard 360, click on Edit > Data Model ① and select Dashboard 360 with filter strategy (leave the filter strategy at Left Join Filter for the moment) ②. You can then click Add Main Search ③, which is the same concept as in ‘Dashboard about a search’.

image

This opens a dialog that enables you to select the appropriate search.

image

In the following screenshot, Companies has been added as the Main Search. By default, the side panel ② shows the list of Dashboard 360-compatible visualizations and whether or not they are assigned to a search node.

At this point, neither of the visualizations are assigned yet (① and ②).

image

To assign a visualization, click the search node to select it; then click the + button ① beside the required visualization, as shown in the following screenshot.

image

We are now going to build the dashboard data model in Siren Investigate by defining how this search is to be relationally connected with other searches (that is, which path in the relational data model to take). To do this, click the + button on the Main Search node (② above).

This opens the following dialog which shows all the possible relations that come from the chosen search, as well as all possible searches that can be reached through those relations. In the case of our demo data, these are the options, and we choose Investments ① via the only possible relation, secured ②.

image

This adds other searches to the dashboard data model so that visualizations can be connected to the appropriate search.

The following screenshot shows how the dashboard data model and the visualizations look after they have been assigned (notice the icons on the top left of each visualization’s container, indicating they have been assigned to the appropriate searches).

image

Once this is done, and the dashboard saved, filtering will work coherently across the visualizations.

Left Filter Join and Inner Filter Join

When ‘Dashboard 360’ is selected, you can specify either a Left Filter Join or an Inner Filter Join.

If Inner Filter Join is selected, this immediately hides from any search all the records that are not fully connected (related) to records in the other searches.

For example, in our demo, if Inner Filter Join is selected and the companies search is connected to the articles, investments, and investors searches, then only 5,615 companies remain; each of these is guaranteed to be connected to at least one article and one investment (which in turn is connected to at least one investor).

image

The Inner Filter Join logic can be too restrictive in many scenarios as it often removes many records from the main search (all those that are not fully relationally connected).

To gain a broader view of the data, the Left Filter Join logic filters the Main Search only if an explicit filter is present on one of the other searches connected to the dashboard datamode.

In our example, selecting Left Filter Join now makes all the companies appear (160,026).

Visualizations that refer to searches other than the Main Search still display only the records that are relationally connected to those in the Main Search. For example in the following screenshot, the pie chart Investment by Year only shows records that are connected to the companies search.

image

Understanding a specific visualization context

When visualizations are part of the Dashboard 360 data model, they display an icon in the top left of their container.

Hovering on this icon shows which node of the dashboard data model they are connected to and the current status of the filters. In the following screenshot, we see that the Investment by Year visualization is relationally filtered to operate on the Investments (41,343 with the current filters) that have been secured by Companies (160,026).

image

Creating a dashboard

Dashboards can be accessed using the home icon or the Siren logo. When you click Dashboard, Siren Investigate displays the first available dashboard or, if no dashboards have been defined, the dashboard creation screen.

New Dashboard screen

You create a new dashboard by clicking the icon in the dashboard panel:

Create New Dashboard

You can give the dashboard a meaningful name and click Create.

Creating a New Dashboard

The dashboard has now been created but it is empty (you will see a message saying This dashboard is empty); you start building the dashboard by adding visualizations to it.

If you have selected fields in Discover (or used its Autoselect fields feature), you can click Generate Dashboard to auto generate a dashboard from those selected fields. This will create a new dashboard with visualizations selected to display the data in those fields. For details, see Auto generate dashboard.

Adding visualizations to a dashboard

Once you have created a new dashboard, you can add visualizations to it.

Click Edit in the toolbar panel — the options change to Editing mode.

Click Add, then select a previously created visualization from the list:

Adding a visualization to the dashboard

You can filter the list of visualizations by typing a filter string into the Visualization Filter field.

The visualization you select appears in a container on your dashboard.

If you see a message about the container’s height or width being too small, resize the container.

If you select another visualization, it appears in another container beside the first one that was added.

Visualizations and searches in a dashboard

A visualization is usually linked to a search. For example, the Article Authors visualization is linked to the Articles search, as is the Articles Wordcloud visualization. Similarly, the Companies Table and Companies Timeline visualizations are linked to the Companies search.

A dashboard can contain visualizations that are all linked to the same search (this provides a logical focus for the dashboard). A dashboard can also contain visualizations that are linked to multiple searches (Dashboard 360). This feature, which was introduced in Siren 10.3, provides great benefits for data analysis.

Before you create a dashboard, you should decide whether it will be based around a single search or multiple searches, as this determines the kind of data model to be used.

  • Single search: Click Data Model, then select the Dashboard is about a search option. You can then select a search from the dropdown list.

  • Multiple searches: Click Data Model, then select the Dashboard 360 with filter strategy option.

When you start with Siren Investigate, it is easier to use dashboards based around a single search. It is very straightforward, and easy to get consistent results.

Before creating a dashboard based around multiple searches (Dashboard 360), you should read The Dashboard Data Model topic, which provides a comprehensive explanation of the feature.

Saving a dashboard

After you have created a dashboard with some visualizations and associated searches, you can save it.

Click Save in the toolbar:

Saving a dashboard

If you want to change the dashboard name from the one it was created with, type it in the Title field.

If Store time with dashboard is checked, the time filter currently set (for example, image) will be restored when the dashboard is reopened.

Click the Save button.

The dashboard is saved, and the name is displayed in the dashboard list on the right.

When you have multiple dashboards created, you can organize them into Dashboard groups.

Customizing the dashboard

Each visualization in your dashboard is stored in a container; you can resize and arrange the containers in the dashboard.

Resizing, moving, and removing containers

Normally only one control (Maximize) is available on the top left of the container’s frame. Clicking this maximizes the container and hides all other containers in the dashboard. Clicking the Restore control returns the visualization to its normal size and position.

To perform other operations on containers, first click Edit on the container’s toolbar.

Moving - Click and hold the container’s Move control to move the container around the dashboard. Other containers will shift as needed to make room for the moving container. Release the mouse button to confirm the container’s new location.

Resizing - Move the cursor to the bottom right corner of the container until the cursor changes to point at the corner. After the cursor changes, click and drag the corner of the container to change the container’s size. Release the mouse button to confirm the new container size.

Removing - Click the x icon in the top right corner of a container to remove that container from the dashboard. Removing a container from a dashboard does not remove the saved visualization in that container.

Viewing detailed visualization information

To display the raw data behind the visualization, click the Spy Open icon at the bottom left of the container. You can select different types of detailed information by selecting an option from the dropdown menu, as in the following examples:

*Table.*

A representation of the underlying data, presented as a paginated data grid. You can sort the items in the table by clicking the table headers at the top of each column.

vis spy table

*Request.*

The raw request used to query the server, presented in JSON format.

Visual spy request

*Response.*

The raw response from the server, presented in JSON format.

Visual spy response

*Statistics.*

A summary of the statistics related to the request and the response, presented as a data grid. The data grid includes the query duration, the request duration, the total number of records found on the server, and the index pattern used to make the query.

Visual spy stats

*Debug.*

A summary of the visualization state (for example, visualization parameters and aggregations) and other details.

Visual spy debug

To export the raw data behind the visualization as a comma-separated-values (CSV) file, click either the Raw or Formatted links at the bottom of the detailed information tabs. A raw export contains the data as it is stored in Elasticsearch. A formatted export contains the results of any applicable Siren Investigate [field formatters].

Working with filters

With your visualizations in place, you can now use them to filter the data. There are several ways to do this:

  • Enter a search term in the Filters field at the top of the visualization.

  • Interact directly with the visualization. For example, click a chart segment - this will apply a filter based on the value selected.

  • Click the Add a filter + link.

When you create a filter, the filter conditions display in an oval under the search text entry box:

filter sample

Moving the mouse pointer over the filter oval displays the following icons:

Filter all buttons.

Enable Filter (image)

Click this icon to switch off the filter without removing it. You can enable the filter again by reselecting the icon. Inactive filters are displayed with a striped background.

Pin Filter (image)

Click this icon to pin a filter. Pinned filters persist across Siren Investigate tabs. You can pin filters from the Visualize tab; when you click the Discover or Dashboard tabs, those filters remain in place.

If you have a pinned filter and you are not seeing any query results, check that your current tab’s index pattern is one that the filter applies to. For example, a filter name:giovanni will results in 0 results if pinned and therefore 'dragged along' to a dashboard whose underlying index does not have a name field, let alone a giovanni value. For this reason, it is good practice in Siren Investigate to use Dashboard Groups to group together dashboards that are based on the same underlying index. In this case, the user can safely pin and 'drag along' a filter across dashboards in the same group.
Toggle Filter (image)

Click this icon to toggle a filter. By default, filters are inclusion filters, and are displayed in green. Only elements that match the filter are displayed. To change this to an exclusion filter, displaying only elements that do not match, toggle the filter. Exclusion filters are displayed in red.

Remove Filter (image)

Click this icon to remove a filter.

Custom Filter (image)

Click this icon to display a text field where you can customize the JSON representation of the filter and specify an alias to use for the filter name, for example, to filter the data to just the companies based in London:

London Companies Filter Example.

Adding the London Companies label to the filter displays that label on the filter bar:

London Companies Filter Bar

Omitting the label displays the filter query in the filter bar:

London Companies Filter Bar

Visualizations listen to filters

Most visualizations in Siren Investigate are connected to a search. The following screenshot shows two two visualizations (a heatmap and an analytic table), both connected to the company search.

image

When a visualization is backed by a search, it ‘listens to’ and reacts to filters or textual queries which are currently on the dashboard.

For example, in the following screenshot, the same visualizations update when a filter is added (in this case, countrycode=USA). Filters can be created either by clicking on the visualizations themselves, or manually with the Add a filter + link.

image

There is a limitation with this simple filtering model, however. All the visualizations will try to apply the filters to their underlying searches, whether or not the filter is applicable.

For example, the countrycode=USA filter will be applied to all the visualizations, even to one that is backed by the Investment search, which does not have a countrycode field. This will cause a 'No results found' on that visualization.

image

For this reason, Investigate dashboards typically group visualizations based on the same search (or searches that have identical/compatible field names), so that filters work coherently across them.

Dashboard 360 overcomes this limitation by allowing relationally connected visualizations described in the Dashboard Data Model section.

JSON filter representation

You can use a JSON filter representation to implement predicate logic, with should for OR, must for AND, and must_not for NOT:

+ .OR Example

{
  "bool": {
    "should": [
      {
        "term": {
          "geoip.country_name.raw": "Canada"
        }
      },
      {
        "term": {
          "geoip.country_name.raw": "China"
        }
      }
    ]
  }
}

+ .AND Example

{
  "bool": {
    "must": [
      {
        "term": {
          "geoip.country_name.raw": "United States"
        }
      },
      {
        "term": {
          "geoip.city_name.raw": "New York"
        }
      }
    ]
  }
}

+ .NOT Example

{
  "bool": {
    "must_not": [
      {
        "term": {
          "geoip.country_name.raw": "United States"
        }
      },
      {
        "term": {
          "geoip.country_name.raw": "Canada"
        }
      }
    ]
  }
}

Click Done to update the filter with your changes.

See Query DSL documentation for more information on the possibilities.

To apply any of the filter actions to all the filters currently in place, click Actions > Global Filter Actions and select an action.

Sharing a dashboard

You can share dashboards with other users by sending a link or by embedding them into HTML pages; ensure that your Siren Investigate installation is properly secured when sharing a dashboard on a public facing server.

To view shared dashboards users must be able to access Siren Investigate; keep this in mind if your Siren Investigate instance is protected by an authentication proxy.

To share a dashboard, click Share to display the Sharing panel.

sharing panel

Click Copy to copy the native URL or embed HTML to the clipboard. The Share Snapshot link field contains a preshortened URL for sharing or embedding.

Embedding a dashboard

Copy the embed code from the Share display into your external web application. Two options are available, iframe with no sidebars and iframe with dashboard navigation sidebar.

Reset all dashboards

You can save a dashboard with specific filters, a custom query, or a certain time range.

remove all filters

If you click Reset Filter  (image) in the toolbar panel, the temporary filters/queries/time range set on all dashboards are removed, and each dashboard reverts to its default state with the saved filters/query/time range.

Dashboard groups

Dashboards can be organized in dashboard groups. Dashboard groups are often used to group together visualizations that are based on the same search.

Dashboard Groups Panel

If the dashboard is associated with a saved search, the count of documents on the dashboard is displayed next to the dashboard name. Two additional indicators that may be displayed are:

  • Filters/Queries indicator: The filter icon is displayed if there are any filters or queries currently applied on the dashboard.

  • Pruned joins indicator: A star symbol is displayed if any join operation was pruned.

You create a dashboard group by clicking the Create new group icon in the dashboard panel.

image

In the dashboard panel, you can change the order of the dashboard groups and move dashboards between groups by dragging and dropping.

Editing a dashboard group

Dashboard groups can be managed by right-clicking the dashboard group name to get the Edit and Delete options.

image

In Edit mode, you can change the title of an existing group, set the icon to a custom image by inserting a URL, or use a https://fortawesome.github.io/Font-Awesome/[Font Awesome] icon.

Auto generate dashboard

If you have selected fields (or used the auto select fields feature) in Discover, you can click Generate Dashboard to auto generate a dashboard from those selected fields. This will create a new dashboard with visualizations selected to display the data in those fields.

First the generated dashboard details are shown and can be edited:

Create auto generated dashboard panel

Here, you can edit the title of the new dashboard, choose whether to store the time with the dashboard (a time filter set on the dashboard will be stored and retrieved when the dashboard is reloaded).

If you want to add a Multichart visualization to the dashboard, select the check box. Be aware that this can slow the generation of the dashboard.

The generated dashboard will be associated to a saved search, which by default is a new saved search created from the current state of the Discover page. If a saved search is currently open in the Discover page, however, you have the option to use it either before or after saving its state.

Generate dashboard report

Clicking OK on the generated dashboard’s details panel opens a panel showing the details of the visualizations to be added to the dashboard:

Generate Dashboard Visualizations Panel

Here, the visualizations to be applied to the dashboard are listed. You can choose whether to add them with the check boxes on the left and you can also edit the automatically generated title of each visualization in the input boxes on the right.

Clicking OK here will save and generate your new dashboard.

New Auto Generated Dashboard

If you want to resize or reorder your visualizations on the dashboard, click Edit on the top bar to begin customizing.

Create a search over all dashboard

Typically, most Siren dashboards have a search bar at the top for drilling down to the dashboard records that match the search.

But in this section we will show how to create a dashboard that can act as a search over all engine, show records that come from any of the Siren accessible indexes, get a preview, and ultimately reach the most appropriate dashboards.

With keyword highlighting, smart field preview selection, a detailed full document panel, and best in class ranking, a Siren search engine dashboard is a must have for most deployments.

Current Limitations

As with Siren 10.0, the method described here work only for data which is stored in the Siren Elasticsearch nodes. Currently, data that is stored outside these nodes, for example accessed directly using Virtual Indexes, is not shown in this search and must instead be searched independently in their own dashboards.

Prerequisite data in the Siren Elasticsearch nodes

The method works for any data available in Siren Elasticsearch nodes. No mandatory method or structure is required. However, consider that if you have control over the way data is loaded, then it is always convenient to have a somewhat similar schema across indexes.

For example, if we know in advance that most records will have a primary time stamp (or a strong concept, for example a "user ID" which is repeated across many indexes) then it is sensible to use the same field name consistently across indexes. This has advantages both in the Search Engine dashboards and across Siren.

In Siren Investigate, this provides consistency for users in the general dashboards when using those attributes, for example creating filters that can be pinned and will work when moved from one dashboard to another. It is also useful in creating an over-all search engine page because it enables you to use widgets (for example an over-all "time stamp" date histogram) that work well across the different indexes.

For the search engine page, you can use a "common denominator" column (in the result table) or drill-down widget to refine the search.

If no fields are purposely the same (have the same type and meaning) across indexes, the search engine will still work and enables you to restrict by content and at a minimum by the name of the originating index.

Consider collision in field names; fields will still be shown but ordering and aggregations on these fields may cause problems.

Configure a * index pattern with exclusions

The next step is to configure a "*" index pattern. When doing this, it is important to add exclusions so that the index pattern will not take indexes which are private to Siren or that might belong to other users of the nodes outside Siren Investigate.

The exclusion pattern can be set in the advanced settings.

image

At this point, ensure you have a saved search associated with this index pattern. In Siren 10, a saved search is created automatically after the index pattern is created. You may want to rename it to "*" or "search over all".

Creating the search over all dashboard

The "search over all" dashboard is a normal dashboard that has the "search over all" saved search as its core dashboard. Here is an example configuration:

image

If you have more shared fields across indexes, for example a time stamp, then more common widgets such as a timeline histogram for the number documents (possibly with a bar split on the "type" of document") could be an interesting addition.

Using the dashboard and user training

Typically, the dashboard works as the user expects, with some caveats:

  • Search uses the same rules as other dashboards. Default Siren configurations use an OR logic, which does not require all words to be present in the result and may be unfamiliar to some users. However, the OR logic is useful because it enables a more expressive query (for example, you can copy and paste a paragraph of text and find documents that contain just a few words from it). Typically, the ranking returns the most significant results first, providing a good overall user experience.

  • To select one or more documents and go to the specific dashboard, you must:

    • Click the check box for the creation of an individual document filter.

    • Select the documents.

    • Click Create Filter, and then pin the filter. This makes it possible to then go to the destination dashboard and see the search results in that environment.

  • The top right time filter has no effect unless a time filter variable is selected on the * index pattern. You should only do so if all the documents have a time stamp field that has the same name or the documents will not be shown.

Search over all and the graph browser

The "search over all" dashboard works well in conjunction with the graph browser. All the results can be added at once using the standard graph browser add document mechanism.

Currently, you should not use "search over all" in conjunction with the relational buttons (pivoting directly from the search interface to the dashboards). This is because with Siren 10, the * pattern is seen as a different entity class altogether in the ontology editor. As such, relations must be recreated and mappings will be very confusing with all the fields mixed together. Also, to go from a search result to its correspondent on the dashboard using a relational button, you would need a field that can `work as primary key (outside the _uid field which cannot be used).

More resources

While the default search quality is good, it can be dramatically improved by taking into consideration the requirements of the application. The following resources are a great place to start improving the quality of your search:

For even more advanced search (semantic, ontology-thesaurus based, multilingual, AI - learn to rank), contact your Siren account manager.

Dashboard color theme

By default, Siren Investigate dashboards use a light color theme. To use a dark color theme instead, click Options (which you can find on the top horizontal menu or by right-clicking the dashboard name) and select the Use dark theme box.

image

You can change the default theme in the Advanced section of the Settings tab.

Refreshing the search results

You can configure a refresh interval to automatically refresh the page with the latest index data. This periodically resubmits the search query.

When a refresh interval is set, it is displayed to the left of the Time Filter in the menu bar.

To set the refresh interval:

  1. Click Time Filter (image).

  2. Click the Refresh Interval tab.

  3. Choose a refresh interval from the list.

To automatically refresh the data, click Auto-refresh (image) when the time picker is open and select an auto refresh interval:

uto refresh intervals

When auto-refresh is enabled, Siren Investigate’s top bar displays a Pause (image) button and the auto-refresh interval.