Run the Understand AI server (ullama)

Every local AI feature — AI Overviews, chat, and AI code search — is served by ullama, a local inference server that ships with Understand (the ai install kits). It is llama.cpp's server behind a small launcher that applies per-model settings. For normal use you never touch it: Understand starts it for you, shares one instance across all your windows, and stops it on exit. This page is for when you want to run, verify, or tune it yourself.

You usually don't need this

Just want AI on? Enable it on the Data page (see overview & setup) and Understand manages the server automatically. Read on only if you're troubleshooting, tuning hardware use, or running the server yourself.

How Understand manages it

When AI is enabled with the bundled Understand Local provider, Understand launches ullama on 127.0.0.1:56767 and talks to it over HTTP using the OpenAI-compatible chat-completions API. Because it binds 127.0.0.1, it is reachable only from your own machine. Multiple Understand windows share the same server, and closing Understand shuts it down.

AI code search needs a second model — a small embedding model — and one server serves one model, so Understand starts a second ullama instance on 127.0.0.1:56768 with --embeddings when the semantic index is in use.

The und ai command also uses the server — it generates AI overviews and the AI Code Search index from the command line, and its -server option can point at an already-running instance.

The executables sit in the Understand program directory next to und — e.g. C:\Program Files\SciTools\bin\pc-win64\ullama.exe on Windows, bin/linux64/ullama on Linux, and inside the app bundle on macOS. Alongside them are ullama-server (the model server itself), ullama-recommend (a hardware sizing tool), ullama-models.conf, and a templates/ directory.

Run it yourself

From that directory you can start the server by hand — useful when verifying an install or running it outside the GUI:

ullama -m /path/to/model.gguf --host 127.0.0.1 --port 56767 -c 32000 --parallel 1

The ai kits install a bundled GGUF under conf/understand/models/; pass that path, or any other GGUF you want to serve.

Start ullama, never ullama-server

ullama is a launcher: it looks at the model filename, applies the matching settings from ullama-models.conf, and then hands off to ullama-server. Those settings are correctness fixes, not preferences — some models produce no tool calls, never stop generating, or fail to start at all without them. Running ullama-server directly skips every one of them.

The launcher prints which rule it matched on startup, so you can see what was applied. Anything you pass yourself wins over the conf file.

Check that it works

While a server is running, query it over HTTP:

• GET http://127.0.0.1:56767/health — readiness. 200 with {"status":"ok"} once the model is loaded; 503 while it is still loading.
• GET http://127.0.0.1:56767/props — the settings actually in effect, including the sampling values the launcher applied.
• GET http://127.0.0.1:56767/v1/models — the model being served.
• GET http://127.0.0.1:56767/slots — per-slot state, useful when requests seem to queue.

To confirm the model really answers, send it a request:

curl http://127.0.0.1:56767/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"In one word, say hi"}]}'

Stop it

Press Ctrl+C in its terminal, or send it a TERM signal (kill <pid> on Linux/macOS) — it shuts down cleanly on both.

Common command-line options

ullama passes everything through to the llama.cpp server, so its full option set is available. The ones that matter most here:

Option	Purpose
-m, --model <file>	The GGUF model to serve. Pass the path as a separate argument, not --model=<path>.
--host <addr>	Interface to bind. 127.0.0.1 (this machine only) or 0.0.0.0 (all interfaces — see Share one AI server).
--port <n>	Port to listen on. Understand uses 56767 for chat and 56768 for embeddings.
-c, --ctx-size <n>	Total context across all slots (see below).
--parallel <n>	Number of request slots.
--embeddings	Serve embeddings instead of chat — required for the AI Code Search model.
-ngl, --n-gpu-layers <n>	Layers to offload to the GPU (-1 = all, 0 = CPU only).
--api-key <key>	Require this key on requests.
--metrics	Enable the Prometheus /metrics endpoint (off by default).

-c is the total, and --parallel divides it

-c 32000 --parallel 2 gives each request 16000 tokens, not 32000. Asking for more context than the machine can hold does not fail at startup — it fails partway through a request, which looks like a broken model rather than a misconfiguration. Size the total for the number of slots you actually want.

Understand itself passes -c 0 --fit-ctx 1024 until you set a Context Window Size: the server then sizes the context to the model and the device it loaded onto, keeping every layer on the GPU and reducing the context only as far as it must. The order matters if you do this by hand — -c 0 on its own tells the server never to reduce the context, so it drops GPU layers instead; --fit-ctx after it is what allows the reduction. Once a size is set, Understand passes it as -c exactly.

Options can also be set through LLAMA_ARG_* environment variables (LLAMA_ARG_MODEL, LLAMA_ARG_CTX_SIZE, LLAMA_ARG_PORT, and so on) — handy when running as a service, where setting environment variables is easier than editing a command line. See Run the AI server as a background service.

Hardware and backends

ullama uses the best backend the bundled build supports: Metal on Apple Silicon, Vulkan on Windows and Linux, and CPU everywhere else. A GPU with ≥ 6 GiB of memory is recommended; -ngl controls how much of the model is offloaded. On macOS, AI runs on Apple Silicon only — it is disabled on Intel Macs.

To size a model to the machine before you download one, run the bundled advisor:

ullama-recommend --json

It reports how much memory is usable, the largest model tier that fits, and a recommended total context to pass to -c.

Speed and answer quality both come from the pairing of hardware and model, not hardware alone — a 1B-parameter model on a laptop GPU and a much larger model on a data-center-grade server will feel like entirely different tools. See Choose a model for the size/VRAM/quality tradeoffs before picking one.

Logs and diagnostics

ullama writes its log to standard error, including the model it loaded, the settings the launcher applied, the backend in use, and the line confirming what it is listening on.

The instance Understand starts in the background discards that output, so when you need to read it, run the server yourself in a terminal with the same model and watch it directly — that is the quickest way to see why a model won't load or which backend was selected. Add -v for verbose logging. If a model won't load, see The AI model file failed to load.