Enable uploads for an Entity Table

Open the entity table in the Data Model page. Click on the Import tab to show the import settings associated with the entity table.

Turn on the Enable in upload center switch to allow users to upload data to this entity table from the upload center. All entity tables have this toggle off by default.

Set the target index

All uploads need a target index where the uploaded records will be stored. You must set the index in the Target index input, which defaults to the entity table’s index pattern. The target index can be an Elasticsearch alias.

After you have set the target index name, Investigate will verify that:

The index is included in the entity table’s index pattern.
The index name resolves to a single index.
The index has the required mappings.

You can’t save the upload settings until all the above conditions are verified.

Target index mappings

The target index requires the following mappings:

{
  "mappings": {
    "properties": {
      "_siren": {
        "properties": {
          "importId": {
            "type": "keyword"
          },
          "importTimestamp": {
            "type": "date"
          },
          "importUser": {
            "type": "keyword"
          },
          "projectId": {
            "type": "keyword"
          },
          "projectImportId": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

Apply the required DLS rules to the index pattern

Make sure that the DLS rule configured in the Set up the required DLS rules section includes the entity table’s index pattern.

This is important to correctly filter the entity table’s content depending on the active data project.

Select target field names

You can turn on the Limit fields for upload switch to select an explicit list of fields that users can upload data to.

This is especially useful to make sure that fields populated by ingestion pipelines are not interfered with.

Adopt a strategy to rotate the target index

You should monitor the target index to make sure that its size doesn’t exceed the recommended threshold. The check should be automated across regular time intervals.

To rotate the index, you can set the Target index to an Elasticsearch alias. When the index size reaches the recommended threshold, create a new index with the same mappings and settings as the current one, then update the alias to target the new index using the Elasticsearch alias API.