Ollama configuration

Ollama is a self-hosted LLM provider. The only required option is the model name. If your instance of Ollama is not running locally, or you are not using the default port, you will need to provide the URL. See here for a full list of supported models.

siren-ai:
  provider: 'ollama'
  providerConfig:
    ollama:
      connection:
        host: 'http://localhost:11434' # Optional. Defaults to the URL of an unconfigured local Ollama instance
      parameters:
        model: 'llama2'

Tool support

You can enable tools for Ollama so users can perform actions such as performing a global search or interacting with the graph by enabling the following parameter:

siren-ai.providerConfig.ollama.parameters.enableTools: true

Please note, effective tool support with Ollama depends heavily on the model you choose. Smaller or older models may fail to call tools correctly, produce malformed responses, or ignore tool instructions entirely.

For best results, choose a model that explicitly supports function/tool calling and has strong instruction-following capabilities. Larger parameter counts generally improve tool reliability, but results will vary. Refer to the Ollama model library for model capabilities and descriptions.

Context window

You can set the context length for Ollama through the CLI using the OLLAMA_CONTEXT_LENGTH environment variable. This should be set on the same server that is running your Ollama instance.

OLLAMA_CONTEXT_LENGTH=32000 ollama serve

A larger value allows the model to consider more information from the input text, which can lead to more coherent and contextually relevant responses. However, increasing this value also increases the computational resources and time required to generate the response, so it is important to balance the context window size with the available resources.

This value defaults to 4096 as it provides a good balance between context size and computational efficiency. Learn more about Ollama context length, see here.