Known limitations

Here is a summary of the known limitations:

  • A join query involving one or more runtime fields is currently supported using HASH_JOIN or BROADCAST_JOIN algorithms. The INDEX_JOIN and ROUTING_JOIN algorithms only support joins with the runtime field on the child set.

  • Regarding fields of nested type:

    • The join filter within a nested query is currently not supported.

    • A nested query within a join filter is supported if and only if the join key does not refer to a field of the nested object.

    • The projection of fields of the nested data type is only possible by using runtime fields. See Joining your data .

  • The projection of fields of runtime lookup data type is currently not supported.

  • A has_child or has_parent query in a join query is supported, but a join query in a has_child or has_parent query is currently not supported.

  • Regarding the _id field:

    • The _id field is supported as a join key field with concrete indices, but its projection as a secondary field is not supported. When the _id field is used as the join key on the child side, no other field can be projected.

    • The _id field is not supported as join key field with virtual indices.

    • The _id field can only be projected as a secondary field using a script or runtime mapping. The script needs to access the stored field of Elasticsearch for the _id, e.g., params._fields._id.value. This access pattern can be slow.

  • The projection of fields in a join with virtual indices isn’t currently supported. However, if both virtual indices are on the same remote cluster with the Federate plugin installed, then it is possible.

  • Spatial joins, as well as multi-conditional joins are limited to the INDEX_JOIN algorithm

Limitations specific to the ROUTING_JOIN

  • The ROUTING_JOIN cannot be selected when joining on something other than the parent set’s _id.

  • The ROUTING_JOIN must not be selected when the parent set’s indices use custom routing as this can result in incorrect joins.

    • When using custom routing, the best practice is to declare the routing value as required in the index mapping, as per Elasticsearch’s documentation.

    • Following this best practice from Elasticsearch will also prevent incorrect join results to be generated inadvertently by Federate.

      • If this best practice is followed, Federate will prompt users with a meaningful error message when they select the ROUTING JOIN algorithm despite the custom routing on the parent set’s indices.

      • If this best practice is followed, Federate’s query planner will never select the ROUTING JOIN in case of custom routing on the parent set’s indices.

  • The ROUTING_JOIN cannot be selected when the parent set’s indices are referred by some alias that uses custom routing for indexing (see Alias routing).

  • The ROUTING_JOIN cannot be selected when the parent set is a data stream with property allow_custom_routing set to true (see _routing field).

Limitations specific to the federate_object runtime field

  • You cannot reference a federate_object runtime field by the sort and join.on clauses.