Page History

Reference for configuring the Local AI endpoint, master enable, tool-category bits, and pointing at alternative LLM endpoints (cloud or alternate local).

AI Integration → Local AI → Configuration

Where the configuration lives

Local AI reads its full configuration from three columns on the existing SolutionSettings table. No new tables, no new columns — one solution has one Local AI configuration.

Editing in the Designer

The Local AI configuration is edited from the Local AI tile on the Data Servers page (sibling of the OPC UA, DataHub, MQTT Broker, and MCP for Runtime tiles).

• Enable Local AI checkbox — toggles ModelEnabled. Master kill-switch.
• Status indicator — reachability probe against the configured endpoint URL. Cached for 30 seconds.
• Endpoint URL — read-only display of the resolved URL.
• Settings link — opens the structured 5-field editor for ModelSettings.
• Model name — read-only display of the configured Name field.

The ModelSettings JSON

Six fields, all defensive — an empty/missing/malformed ModelSettings column transparently resolves to defaults. Unknown extra keys are preserved across edit cycles, so future revisions stay forward-compatible.

{
  "URL": "http://localhost:11434/v1/chat/completions",
  "Name": "qwen2.5:7b-instruct",
  "Authorization": "NoAuth",
  "Headers": "",
  "Info": "Default local model. Apache 2.0, ~4.7 GB.",
  "TimeoutSeconds": 60
}

The configuration is parsed defensively on every Local AI call — the parse cost is negligible compared with the LLM round-trip, and there is no caching layer to invalidate when the JSON changes.

The ModelOptions bitmask

An integer column carrying independent enable bits. The bitmask is shared with the AI Runtime Connector and the AI Designer connector — the same bits gate the same tool categories regardless of which transport the LLM uses to call them.

The five tool-category bits (0x04–0x20) are AND-gated against the master bit. A category bit ON without the master bit ON leaves the category effectively OFF.

Master gate order

Both consumer paths apply the gates in a fixed order:

1. ModelEnabled — if false, return status="disabled" immediately. No HTTP traffic. latencyMs = 0.
2. ModelOptions bit 0x02 — ChatRequest only: if the master tool-surface bit is OFF, return status="disabled". TK.AIExecute skips this gate (no tools to expose).
3. Per-category bits — ChatRequest only: AND-ed against the master bit when assembling the tool catalog the LLM sees during a chat turn.

Pointing at a different LLM endpoint

Replace the URL and Name fields. Any OpenAI-compatible chat-completions endpoint works.

Local Ollama (default)

{
  "URL": "http://localhost:11434/v1/chat/completions",
  "Name": "qwen2.5:7b-instruct",
  "Authorization": "NoAuth",
  "Headers": ""
}

Remote Ollama on a GPU server

{
  "URL": "http://gpu-server-01.lan:11434/v1/chat/completions",
  "Name": "qwen2.5:14b-instruct",
  "Authorization": "NoAuth",
  "Headers": ""
}

The remote Ollama must be started with OLLAMA_HOST=0.0.0.0:11434 and the firewall opened on TCP 11434.

OpenAI-compatible cloud endpoint with Bearer token

{
  "URL": "https://api.example-llm-provider.com/v1/chat/completions",
  "Name": "the-cloud-model-name",
  "Authorization": "BearerToken\n/secret:CloudLLMApiKey",
  "Headers": ""
}

The /secret:CloudLLMApiKey token resolves at call time from the SecuritySecrets vault — the actual API key never appears in the configuration. See SecuritySecrets Authentication for Local AI.

Endpoint with extra HTTP headers

Some providers require extra request headers (organization ID, project ID, region). Add them via the Headers field, one Key: Value pair per line:

{
  "URL": "https://api.example-llm-provider.com/v1/chat/completions",
  "Name": "the-cloud-model-name",
  "Authorization": "BearerToken\n/secret:CloudLLMApiKey",
  "Headers": "X-Organization-Id: org-12345\nX-Project-Id: /secret:CloudLLMProjectId"
}

Header values also accept /secret:<Name> tokens.

Configuration safety nets

The platform applies several safety nets to prevent silent misconfiguration:

• Defensive defaults. Empty / null / malformed ModelSettings falls back to the recommended local Ollama defaults. A solution with a corrupted JSON blob still works against the local default.
• Status probe. The Local AI tile in the Designer probes the resolved URL on a 30-second cache, surfacing a red indicator when the endpoint is unreachable. Use it before deploying.
• Master kill-switch precedes everything. A solution can be staged with full configuration and shipped with ModelEnabled = false. No LLM traffic flows until the customer toggles it ON.
• Off-server short-circuit. Secret resolution is a server-side operation. Calls reaching Local AI from a thin-client context cannot resolve secrets and fall through to a normal HTTP error reply — no silent unauthenticated POST.

What this page does NOT cover

• Bringing up Local AI on a fresh machine. See Local AI - First Install Walkthrough.
• SecuritySecrets reference syntax and examples. See SecuritySecrets Authentication for Local AI.
• Reply envelope shape and status semantics. See Local AI Reply Envelope Schema.
• The ChatRequest Display action. See ChatRequest Action Reference.
• The TK.AIExecute script API. See TK.AIExecute API Reference.

In this section...