Configuring the off-heap memory
To prepare your system for use, you must first configure the off-heap memory.
The memory module is responsible for allocating and managing chunks of off-heap memory. For more information, see the Memory section of the Modules topic.
Siren Federate provides a cluster-level API that allows you to retrieve statistics about the cluster and the off-heap memory allocation.
The allocated direct memory represents the off-heap memory chunks pre-allocated to accommodate the root allocator.
The chunk of off-heap memory that was allocated is kept and reused, because off-heap memory allocation is expensive.
The root allocator can then allocate off-heap memory buffers of various size in a very efficient way.
You configure the amount of off-heap memory available for the root allocator by updating the
siren.memory.root.limit variable in the
However, this value is limited by the maximum direct memory limit of Siren Federate.
To set a larger limit for the root allocator, you must first manually set the maximum direct memory limit for Siren Federate.
config/jvm.optionsfile in your Elasticsearch instance.
siren.io.netty.maxDirectMemoryvariable. For example, to increase the maximum direct memory limit to 2GB, add the following line:
config/elasticsearch.ymlfile and set the
siren.memory.root.limitparameter to a value that is less than the value you set in step 2, for example, 2147483647.
It is forbidden to use a limit that is greater than or equal to the maximum direct memory limit.
Start the Elasticsearch instance. The following info logs are displayed:
[2019-12-10T17:29:11,207][INFO ][i.s.f.c.i.m.BufferAllocatorService] [node_s0] Buffer allocator service starting with Unsafe access: true [2019-12-10T17:29:11,207][INFO ][i.s.f.c.i.m.BufferAllocatorService] [node_s0] Buffer allocator service starting with directMemoryLimit=2147483648 (1) [2019-12-10T17:29:11,233][INFO ][i.s.f.c.i.m.BufferAllocatorService] [node_s0] Buffer allocator service starting with defaultNumDirectArenas=5 [2019-12-10T17:29:11,236][INFO ][i.s.f.c.i.m.BufferAllocatorService] [node_s0] Instantiating root allocator with limit=2147483647 (2)
These info logs provide an overview of how Siren Federate is configured. Here, we can see that:
1 The maximum direct memory limit is correctly set to 2147483648. 2 The root allocator limit is correctly set to 2147483647.
It is critical to ensure that there is enough available memory on the machine to accommodate the maximum direct memory limit for Siren Federate, the JVM maximum heap memory limit, and the operating system.
If the sum of maximum direct memory limit for Siren Federate and the JVM maximum heap memory limit does not leave enough memory for the operating system, the OS might stop the Elasticsearch instance (OOM killer process on Linux systems).
The following are the recommended settings for a cluster that needs to execute joins on a large amount of data:
24 GB heap for Elasticsearch
24 GB off-heap for Siren Federate
16 GB for the operating system and OS cache
Alternatively, if the off-heap memory for Siren Federate is not fully used, it is better to give more heap memory to Elasticsearch:
32 GB heap for Elasticsearch
16 GB off-heap for Siren Federate
16 GB for the operating system and OS Cache
32 GB heap for Elasticsearch
64 GB off-heap for Federate
32 GB for the operating system and OS Cache
Alternatively, if the off-heap memory for Siren Federate is not fully used, it is better to give more heap memory to the operating system and the OS cache.