Management

An index pattern search identifies one or more Elasticsearch indices that you want to explore with Siren Investigate. Siren Investigate looks for index names that match the specified pattern. An asterisk (*) in the pattern matches zero or more characters. For example, the pattern myindex-* matches all indices whose names start with myindex-, such as myindex-1 and myindex-2.

An index pattern search can also be the name of a single index.

To create an index pattern search to connect to Elasticsearch:

Note that the colon ':' has been deprecated in index names, and should not be used.

To use an event time in an index name, enclose the static text in the pattern and specify the date format using the tokens described in the following table.

For example, [logstash-]YYYY.MM.DD matches all indices whose names have a timestamp of the form YYYY.MM.DD appended to the prefix logstash-, such as logstash-2015.01.31 and logstash-2015-02-01.

The default index pattern search is loaded automatically when you view the Discover tab. Siren Investigate displays a star to the left of the name of the default pattern in the Index Pattern Search list in the Data model app. The first pattern you create is automatically designated as the default pattern.

To set a different default index pattern search:

When you add an index mapping, Siren Investigate automatically scans the indices that match the pattern to display a list of the index fields. You can reload the index fields list to pick up any newly-added fields.

Reloading the index fields list also resets Siren Investigate’s popularity counters for the fields. The popularity counters keep track of the fields you have used most often within Siren Investigate and are used to sort fields within lists.

To reload the index fields list:

Elasticsearch supports the ability to run search and aggregation requests across multiple clusters using a module called cross-cluster search.

To take advantage of cross-cluster search, you must configure your Elasticsearch clusters accordingly. Refer to the corresponding Elasticsearch documentation before attempting to use cross-cluster search in Siren Investigate.

After your Elasticsearch clusters are configured for cross-cluster search, you can create specific index patterns in Siren Investigate to search across the clusters of your choosing. Using the same syntax that you would use in a raw cross-cluster search request in Elasticsearch, create your index pattern in Siren Investigate with the convention <cluster-names>:<pattern>.

For example, if you want to query logstash indices across two of the Elasticsearch clusters that you set up for cross-cluster search, which were named cluster_one and cluster_two, you would use cluster_one:logstash-*,cluster_two:logstash-* as your index pattern in Siren Investigate.

Just like in raw search requests in Elasticsearch, you can use wildcards in your cluster names to match any number of clusters, so if you wanted to search logstash indices across any clusters named cluster_foo, cluster_bar, and so on, you would use cluster_*:logstash-* as your index pattern in Siren Investigate.

If you want to query across all Elasticsearch clusters that have been configured for cross cluster search, then use a standalone wildcard for your cluster name in your Siren Investigate index pattern: *:logstash-*.

After an index pattern is configured using the cross-cluster search syntax, all searches and aggregations using that index pattern in Siren Investigate take advantage of cross-cluster search.

Siren Federate provides three types of join algorithms. The plugin tries to pick the best algorithm for a given join automatically. However, you can force the selection by choosing one of the available options:

A detailed description of each algorithm can be found in the Siren Federate plugin documentation.

For time based indices, the maximum time range can be defined. Setting a longer time range will be prevented by the system and a warning will be displayed.

The maximum number of documents that can be displayed on a dashboard can be set. If the filters are changed to show a larger number of documents, the system will not allow the operation and show a warning.

If you attempt to make a join between two dashboards by using the relational navigator and the number of documents on one or both of the dashboards exceeds the limit, the navigation is prevented and a warning message is displayed.

String fields support the String and URL formatters.

The String field formatter can apply the following transformations to the field’s contents:

The URL field formatter can take on the following types:

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Date fields support the Date, Url, and String formatters.

The Date formatter enables you to choose the display format of date stamps using the moment.js standard format definitions.

The String field formatter can apply the following transformations to the field’s contents:

The URL field formatter can take on the following types:

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Geographic point fields support the String formatter.

The String field formatter can apply the following transformations to the field’s contents:

Numeric fields support the URL, Bytes, Duration, Number, Percentage, String, and Color formatters.

The URL field formatter can take on the following types:

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

The String field formatter can apply the following transformations to the field’s contents:

The Duration field formatter can display the numeric value of a field in the following increments:

You can specify these increments with up to 20 decimal places for both input and output formats. The default number of decimals for the Number format is 3, i.e. 0,0.[000]. If there are values smaller than this, but larger than 1e-7, they will be rounded to 0. The fix is to change the Numeral.js format pattern to: 0,0.[0000000]

The Color field formatter enables you to specify colors with specific ranges of values for a numeric field.

When you select the Color field formatter, Siren Investigate displays the Range, Font Color, Background Color, and Example fields.

Click Add Color to add a range of values to associate with a particular color. You can click in the Font Color and Background Color fields to display a color picker. You can also enter a specific hex code value in the field. The effect of your current color choices are displayed in the Example field.

The Bytes, Number, and Percentage formatters enable you to choose the display formats of numbers in this field using the numeral.js standard format definitions.

Scripted fields compute data on the fly from the data in your Elasticsearch indices. Scripted field data is shown on the Discover tab as part of the document data, and you can use scripted fields in your visualizations. Scripted field values are computed at query time so they are not indexed and cannot be searched. Note that Siren Investigate cannot query scripted fields.

When you define a scripted field in Siren Investigate, you have a choice of scripting languages. Starting with 5.0, the default options are Lucene expressions and Painless. While you can use other scripting languages if you enable dynamic scripting for them in Elasticsearch, this is not recommended because they cannot be sufficiently sandboxed.

You can reference any single value numeric field in your expressions, for example:

For more background on scripted fields and additional examples, refer to Using Painless in Kibana scripted fields.

Creating a scripted field

For more information about scripted fields in Elasticsearch, see Scripting.

Modifying a scripted field

Deleting a scripted field