Data reflection

There are 5 steps in defining a new datasource reflection job:

Drag and drop file into import panel or select a file to import from the file browser. When file is uploaded, select the sheet to import. If there are is a previously defined configuration for the data you wish to import, this will be used in help in step 2.

To configure the import of a file, follow these steps:

The Neo4j Import Wizard (beta) makes it easy to import Neo4j data from a datasource and then configure and view the data model.

Go to Management > Datasources.

In the New Datasource Type dropdown, click JDBC.

Specify the details for the Neo4j as the datasource, then click Save.

Go to Data Reflections and click Reflection Jobs; add a new job.

Select the Neo4j as the datasource; you then get an option to use the Neo4j Importer.

Click Use Neo4j Importer. The Neo4j Import Wizard screen opens.

Select the required nodes from the Select Nodes dropdown, and click Next.

All data reflection jobs for nodes and relations are displayed. Note that relations are also listed under Node Name; this is because Neo4j relations contain data, and Siren Investigate runs a reflection job for each relation, just like a node.

Click Confirm.

Wait for the data to start reflecting, which means that all jobs have indexed at least one document, indicating that the fields are mapping successfully. The Continue button becomes enabled at this point.

While a job is indexing, its status is shown as running; when a job completes, the status changes to successful.

You can leave the wizard at this point (by clicking ../ on top left of the wizard, or anywhere else in the application). This may be necessary if reflection jobs are failing and you have to go back to fix them.

You can continue where you left off by following the notification in the Datasource Reflection Jobs page. Click Pending Jobs.

Click Continue Neo4j Job.

On the Configure Saved Searches screen, you can modify the search name, and specify a color for the nodes and relations:

Click Next.

On the Configure Relations screen, you can modify the Relation Label and the Inverse Relation Label for each relation:

Click Create Ontology.

You can now go to the Data Model page to view the Neo4j data model:

Node behavior

You should note the following aspects of node behavior:

Other considerations

The following steps add a 'Shortest Path' query script to the Graph Browser to find the shortest path between two or more (Neo4j reflected) data nodes.

Use the Neo4j Import Wizard (beta) to import your Neo4j data.

Locate the Shortest Path script in the demo data supplied with Siren 10.3.1; alternatively, you can find the script at this location.

Navigate to Management > Scripts ① and create a new script.

Enter a title and description for the script ②.

Select contextual from the Type drop-down list ③.

Paste the script into the Source section ④.

Click Save ⑤.

Go to the Graph Browser and click the Edit button on the top panel.

On the Options tab on the left, click the Add Contextual Script button ① under the Contextual Function.

Select the newly created script from the drop-down list ② to add it to the Graph Browser.

Click Save ③.

Running the Shortest Path script

You can now use this script to compute shortest paths between your selected (Neo4j reflected) data nodes.

Add the Neo4j reflected nodes to the Graph Browser.

Select the required nodes ①.

Right-click and select Neo4j Shortest Path (the name of the script) ②.

In the dialog that opens, enter the maximum path length and click OK (both of the fields will be already populated).

The shortest path between the nodes is displayed.

The following limitations apply to running a Neo4j Shortest Path script:

Split Fields:

To split a string, separated by delimiter "|" into a list of sub-strings, and if no initial string exists, fill the target field with an empty string

Split Fields to a long:

To accomplish a similar goal, but this time convert each sub-string to a long, and if no value exists in the initial field, on failure set the target field to -1.

To extract text and create a new field (Using regex):

To extract the text between the first set of parentheses in the Title field and create a new field Patent_ID for it.

Merge two field to create a geo_point:

Merge two fields providing 'latitude' and 'longitude' to create a single Elasticsearch geo_point field:

Cron-Expressions are used to configure schedules. Cron-Expressions are strings that are actually made up of seven sub-expressions, that describe individual details of the schedule. These sub-expression are separated with white-space, and represent:

Individual sub-expressions can contain ranges and/or lists.

Wild-cards (the ‘‘ character) can be used to say “every” possible value of this field. Therefore the ‘‘ character in the “Month” field of the previous example simply means “every month”. A ‘*’ in the Day-Of-Week field would therefore obviously mean “every day of the week”.

All of the fields have a set of valid values that can be specified. These values should be fairly obvious – such as the numbers 0 to 59 for seconds and minutes, and the values 0 to 23 for hours. Day-of-Month can be any value 1-31, but you need to be careful about how many days are in a given month! Months can be specified as values between 0 and 11. Days-of-Week can be specified as values between 1 and 7 (1 = Sunday).

The ‘/’ character can be used to specify increments to values. For example, if you put ‘0/15’ in the Minutes field, it means ‘every 15th minute of the hour, starting at minute zero’. If you used ‘3/20’ in the Minutes field, it would mean ‘every 20th minute of the hour, starting at minute three’ – or in other words it is the same as specifying ‘3,23,43’ in the Minutes field. Note the subtlety that “/35″ does *not mean “every 35 minutes” – it mean “every 35th minute of the hour, starting at minute zero” – or in other words the same as specifying ‘0,35’.

The ‘?’ character is allowed for the day-of-month and day-of-week fields. It is used to specify “no specific value”. This is useful when you need to specify something in one of the two fields, but not the other.