Choose a model (local LLM or cloud provider)¶
Understand ships with several local models and can also use external runtimes (LM Studio, Ollama) or cloud providers. There's no single "best" model — only the best fit for your hardware, your codebase, and any compliance rules you work under.
Open the picker from Data preferences → the ... button next to the AI Provider row.
The picker checks your machine
Understand runs the bundled ullama-recommend when it builds this list, and marks the entry
it recommends for your computer. Models your machine cannot hold are labeled (more memory
than this computer has) — still selectable, in case you know better or are about to upgrade,
but flagged so the choice is deliberate. "Hold" means the model's weights and a working
context together: each model's context memory is costed at its own rate, so a sliding-window
model such as Gemma4-E4B fits on a card that a size-only rule would rule it out of. The
recommendation moves down to a smaller model on a constrained machine; it does not push you to
a larger one just because it fits, because the larger models are several times slower per
overview and overview generation runs in the background by default.
Picking a model also fills in Context Window Size with the context that model can use on this computer — the memory left after its weights, divided by what one token of its context costs. The number stays yours to edit: the field caps at the window the model was trained for, and a value above the estimate is allowed with a warning that it may run slowly or fail. On a machine with a discrete GPU the estimate tracks what the server itself settles on — within a few percent in SciTools' measurements.
Quick recommendation¶
| If you… | Use |
|---|---|
| Want a good all-round default | Granite4.1-3B — the recommended model |
| Have very little memory, and can accept weaker answers | Qwen3-1.7B |
| Have little memory, but want answers that clear SciTools' quality bar | Qwen3.5-2B |
| Want instructions followed most reliably | Gemma4-E4B |
| Want the best chat answers and the most thorough investigation of the code, and have a workstation | Qwen3.6-35B-A3B |
| Must use US-origin software | Granite4.1-3B (IBM), any Gemma (Google), or gpt-oss-20b (OpenAI) |
Bundled local models¶
The model picker offers a range of models, in increasing size. On Windows, the installer downloads the model you pick during installation (Granite by default — Advanced Install chooses another or none). Beyond that, missing models are downloaded on demand (from SciTools' Hugging Face page) the first time you select them.
- Qwen3-1.7B — 1.1 GB. The fastest responses and the smallest footprint. It is the only bundled model that falls short of SciTools' quality bar, so choose it only when memory is tight. Alibaba.
- Qwen3.5-2B — 1.3 GB. Very fast at generating overviews in bulk, and its chat answers clear SciTools' quality bar, though they trail Granite4.1-3B's. Alibaba.
- Granite4.1-3B — 2.2 GB. Recommended: in SciTools' testing its chat answers match the code more closely than any other model's except Qwen3.6-35B-A3B, and it generates overviews in bulk as fast as any model here, though some larger models write more accurate ones. Qwen3-1.7B replies more than twice as fast, with much weaker answers. IBM, US-origin, Apache-2.0.
- Gemma4-E2B — 3.1 GB. Runs on very little graphics memory (only about 1.5 GB of it loads onto the card), and its chat answers are close to Qwen3.5-2B's. It thinks before answering in chat, which makes its chat replies slower. Google, US-origin, Apache-2.0.
- Gemma4-E4B — 4.2 GB. Follows instructions most reliably of the models measured, and its chat answers match the code about as closely as Gemma4-12B's, but it is three to four times as slow to respond as Granite and much slower at bulk overviews. Google, US-origin, Apache-2.0.
- Qwen3.5-9B — 5.7 GB. Strong chat answers, close to Granite4.1-3B's, and it reads more of the code before answering. But it is about twice as slow to respond. Alibaba.
- Gemma4-12B — 7.1 GB. Its chat answers match the code about as closely as Gemma4-E4B's, but it is the slowest model here. Needs a high-end machine: about 8 GB of VRAM, or an Apple Silicon machine with plenty of unified memory. Google, US-origin, Apache-2.0.
- gpt-oss-20b — 11.6 GB. Quick for its size, because it activates only a fraction of its weights per token, though it is slow at bulk overviews. Its chat answers match the code about as closely as the Gemma models'. Needs a high-end machine: about 12 GB of VRAM, or an Apple Silicon machine with plenty of unified memory. OpenAI, US-origin, Apache-2.0.
- Qwen3.6-35B-A3B — 22.4 GB. The best chat answers in SciTools' testing, and it reads the most code before answering. It is fast for its size because it activates only a fraction of its weights per token, but it needs a workstation-class GPU (about 22 GB of VRAM) or a large Apple Silicon machine. Alibaba.
A bigger model is not a better one
In SciTools' testing, only Qwen3.6-35B-A3B's chat answers matched the code more closely than Granite4.1-3B's. Several of the models in between read more of the code, and their answers were still less accurate. They are also slower, so choose one of them for what its entry says it does well, not for its size.
SciTools' test results
Each model answered 16 questions about a real C++ project and wrote 39 code overviews, with Understand's default settings on an Nvidia RTX 4090. Scores run from 0 to 1, and higher is better. A model's scores can move by up to 0.1 between runs, so treat small differences as ties. Reply time is the median for one question. Times are from that card, and a slower machine takes longer.
| Model | Answers | Reading | Instructions | Overviews | Reply time | Overviews/hr |
|---|---|---|---|---|---|---|
| Qwen3-1.7B | 0.57 | 0.70 | 0.76 | 0.69 | 2 s | 8,100 |
| Qwen3.5-2B | 0.71 | 0.77 | 0.84 | 0.66 | 4 s | 9,900 |
| Granite4.1-3B | 0.77 | 0.81 | 0.85 | 0.64 | 4 s | 10,100 |
| Gemma4-E2B | 0.71 | 0.77 | 0.83 | 0.65 | 11 s | 5,100 |
| Gemma4-E4B | 0.68 | 0.82 | 0.88 | 0.67 | 16 s | 3,800 |
| Qwen3.5-9B | 0.75 | 0.83 | 0.83 | 0.66 | 9 s | 3,700 |
| Gemma4-12B | 0.65 | 0.78 | 0.83 | 0.68 | 24 s | 2,300 |
| gpt-oss-20b | 0.70 | 0.81 | 0.85 | 0.69 | 8 s | 3,300 |
| Qwen3.6-35B-A3B | 0.78 | 0.89 | 0.86 | 0.67 | 11 s | 4,500 |
VRAM guidance¶
The table below shows the peak graphics memory each bundled model took in SciTools' testing, on Linux with an Nvidia RTX 4090, with the context Understand chose there: 16,384 tokens. A larger context takes more, and how much more depends on the model. Understand also starts a small embeddings model alongside, which adds about 0.1 GB.
| Model | Graphics memory | Also in system RAM |
|---|---|---|
| Qwen3.5-2B | 1.6 GB | |
| Gemma4-E2B | 1.9 GB | about 1.6 GB of per-layer data |
| Qwen3-1.7B | 3.1 GB | |
| Gemma4-E4B | 3.1 GB | about 1.6 GB of per-layer data |
| Granite4.1-3B | 3.6 GB | |
| Qwen3.5-9B | 5.9 GB | |
| Gemma4-12B | 8.1 GB | |
| gpt-oss-20b | 11.8 GB | |
| Qwen3.6-35B-A3B | 22.3 GB |
A model's memory is not its download size: Qwen3-1.7B is the smallest download but needs more memory per token of context than Qwen3.5-2B or the Gemma models. Every model also keeps its token embeddings (0.2 to 0.6 GB) in system RAM rather than on the card, and the Gemma E-series keep about 1.6 GB of per-layer data there too. When in doubt, start with the recommended model.
Size it against your own machine
The bundled ullama-recommend tool reads the machine's memory — and on Windows and Linux the
free memory of a discrete GPU — and reports the largest model size and parameter tier that will
fit, a suggested model, and a recommended total context. Give it a model file and it also reports
the context this machine can run that model with:
ullama-recommend --json
ullama-recommend --model /path/to/model.gguf
Run it from the Understand program directory (see Run the Understand AI server). It accounts for your actual hardware, which a static table cannot. For a sliding-window model the per-token cost it reads from the file is an upper bound, so the context it reports for one is a floor.
Bring your own GGUF¶
In the model picker choose Custom GGUF file and browse to any compatible .gguf model —
useful when your organization has standardized on a specific model. You can also set the context
window size for local models.
The bundled server tracks current llama.cpp, so recent model families load as they are released, and it carries per-model settings for the ones that need them. Families it recognizes today include Qwen3, Qwen3-Coder, Qwen3.5 and Qwen3.6, Gemma 3 and Gemma 4, Granite 4, Phi-4-mini, GPT-OSS, Mistral Small, Devstral, Nemotron 3.5 Lightning, Cohere North Mini Code, Apriel, and DeepSeek-R1 distills. Anything else falls back to the model's own built-in settings.
Those per-model settings matter
Some models simply do not work without them: Phi-4-mini emits no tool calls under its own chat template, Apriel's template stops the server from starting, and the Qwen3.5 models never stop reasoning on summarization tasks. The server applies the right fix based on the model's filename, so keep the published filename when you copy a GGUF into place — a renamed file falls back to defaults, and the launcher's startup log says when no rule matched.
Use an external runtime or a cloud provider¶
The Provider dropdown offers, besides the bundled local model:
Custom server context window
When you choose Other (a custom OpenAI-compatible URL), Understand cannot query the server for its context window the way it does with LM Studio and Ollama. A Context Window Size field appears for that provider only — set it to the token limit your server runs (default 32768). This lets Understand budget and summarize prompts before sending them, instead of letting the server decide what to do with an oversized request.
For all cloud providers (ChatGPT, Claude, Gemini, Grok, and custom endpoints), overview prompts are trimmed to fit the configured context window before being sent, so a large file's overview is shortened rather than refused outright.
| Provider | Endpoint (default) | API key |
|---|---|---|
| LM Studio | 127.0.0.1:1234 |
not required |
| Ollama | 127.0.0.1:11434 |
not required |
| ChatGPT (OpenAI) | api.openai.com |
required |
| Claude (Anthropic) | api.anthropic.com |
required |
| Gemini (Google) | Google endpoint | required |
| Grok (xAI) | api.x.ai |
required |
| Other | your OpenAI-compatible URL | as configured |
Cloud providers send your prompts off the machine
LM Studio and Ollama still run locally. The cloud providers (ChatGPT, Claude, Gemini, Grok, or a custom endpoint) send prompts — including the code context in them — to that third party. An API key is required, HTTPS is enforced, and you must tick the data-sharing and fee acknowledgements before the setting is accepted. See What can the AI access?, and for a deployment that must guarantee no cloud calls at all, see Local AI with no cloud calls.
For LM Studio and Ollama setup — including serving one model to a whole team from a machine on your network — see Set up a more robust local AI server.