Choose a model (local LLM or cloud provider)¶
Understand ships with several local models and can also use external runtimes (LM Studio, Ollama) or cloud providers. There's no single "best" model — only the best fit for your hardware, your codebase, and any compliance rules you work under.
Open the picker from Data preferences → the ... button next to the AI Provider row.
Quick recommendation¶
| If you… | Use |
|---|---|
| Just want a good default | Qwen3-1.7B (the built-in default) |
| Want stronger reasoning and have the RAM | Qwen3-4B |
| Work in large/complex codebases | Qwen3-8B |
| Must use US-origin software | Granite4-Micro (IBM) |
Bundled local models¶
Missing models are downloaded on demand (from SciTools' Hugging Face page) the first time you select them.
- Qwen3-1.7B — the default. Best balance of quality, speed, and size for most workflows.
- Qwen3-4B — noticeably better reasoning and context retention.
- Qwen3-8B — strongest cross-file reasoning; slower and needs more VRAM.
- Granite4-Micro (3B parameters) — IBM, US-origin. Choose this when regulatory, contractual, or internal policy requires a US-developed model.
Why Qwen is the default
In SciTools' internal testing, Qwen models were consistently stronger than the older Llama models for code-understanding tasks — explaining unfamiliar code, following multi-step instructions, and staying grounded instead of hallucinating. Older Llama 3.2 models are retained only for legacy builds.
VRAM guidance¶
Larger models need more VRAM. As a rough guide from SciTools' benchmarking, a small model (≈1–2B) fits comfortably in ~4–6 GB, a mid model (≈4B) wants ~8 GB, and an 8B model wants ~11 GB+ on discrete GPUs. Apple unified-memory and integrated GPUs use less but run slower. When in doubt, start small and move up only if you rely heavily on AI explanations.
The table below shows approximate peak VRAM (unified memory on Apple ARM) that SciTools observed in benchmarking, per model and hardware type. Figures are approximate and hardware-dependent — treat them as a starting point, not a guarantee. Blank cells were not tested on that configuration; not every model was measured on every setup.
| Model | Nvidia GPU (Windows) | AMD GPU (Windows) | Intel iGPU (Windows) | Nvidia GPU (Linux) | Unified memory (Apple ARM) |
|---|---|---|---|---|---|
| Qwen3-0.6B¹ | ~5 GB | ~5.1 GB | — | — | ~3.7 GB |
| Qwen3-1.7B | ~5.6 GB | ~5.8 GB | ~6 GB | ~5.8 GB | ~3.7 GB |
| Qwen3-4B | — | — | ~8.6 GB | ~8.1 GB | ~4.8 GB |
| Qwen3-8B | — | — | — | ~11.2 GB | ~8.8 GB |
| Granite 4.0-1B¹ | — | — | — | ~4.6 GB | ~2.7 GB |
| Granite4-Micro (3B) | ~6.3 GB | — | ~6.7 GB | ~7.2 GB | ~4.1 GB |
| Llama 3.2-1B | ~3.3 GB | — | ~4.6 GB | — | ~2.2 GB |
| Llama 3.2-3B | — | ~8.1 GB | ~8.7 GB | — | ~5 GB |
| Llama 3.2-8B | — | — | ~11.6 GB | — | ~9.5 GB |
¹ Benchmark-only rows: the in-app model picker offers exactly four models — Qwen3-1.7B, Qwen3-4B, Qwen3-8B, and Granite4-Micro (3B). Qwen3-0.6B and Granite 4.0-1B require the Custom GGUF route. The Llama 3.2 models ship only with legacy builds of Understand.
Bring your own GGUF¶
In the model picker choose Custom GGUF file and browse to any compatible .gguf model —
useful when your organization has standardized on a specific model. You can also set the context
window size for local models.
Use an external runtime or a cloud provider¶
The Provider dropdown offers, besides the bundled local model:
| Provider | Endpoint (default) | API key |
|---|---|---|
| LM Studio | 127.0.0.1:1234 |
not required |
| Ollama | 127.0.0.1:11434 |
not required |
| ChatGPT (OpenAI) | api.openai.com |
required |
| Claude (Anthropic) | api.anthropic.com |
required |
| Gemini (Google) | Google endpoint | required |
| Grok (xAI) | api.x.ai |
required |
| Other | your OpenAI-compatible URL | as configured |
Cloud providers send your prompts off the machine
LM Studio and Ollama still run locally. The cloud providers (ChatGPT, Claude, Gemini, Grok, or a custom endpoint) send prompts — including the code context in them — to that third party. An API key is required, HTTPS is enforced, and you must tick the data-sharing and fee acknowledgements before the setting is accepted. See What can the AI access?, and for a deployment that must guarantee no cloud calls at all, see Local AI with no cloud calls.
For LM Studio and Ollama setup — including serving one model to a whole team from a machine on your network — see Set up a more robust local AI server.