Can a team run Mistral Vibe without sending code to a hosted model? Yes. The CLI talks to any OpenAI-compatible server, and Mistral documents the settings for a fully offline setup. This guide covers what Mistral documents for servers and hardware, then the setup problems users reported.
When you need this
- Code may not leave the developer's machine or the company network.
- You want to cap spend at the hardware you already own.
- Engineers already serve open-weight models with vLLM, llama.cpp, LM Studio, or Ollama.
- You want a fallback for days the hosted service is slow or the plan budget is spent.
Evaluation criteria
Pick the server
Which local server will Vibe call?
Look forvLLM, which Mistral recommends, or llama.cpp, LM Studio, or Ollama, each behind an OpenAI-compatible endpoint set as a provider in config.toml. A built-in llamacpp preset points at 127.0.0.1:8080 with no key.
Size the model to the hardware
Which open-weight model fits the GPUs you have?
Look forDevstral Small 2 is a dense 24B model. Run at 4-bit with a 32k context on a 24 GB GPU such as an RTX 4090, or FP8 with about 128k context on an H100 or A100. Mistral Small 4 needs at least one H100 80GB at FP8. One user ran Devstral Small 2 at its full 256k context on three cards. Another got working C++ changes from it.
Test tool calls before rollout
Does the model call Vibe's tools correctly on your server?
Look forA trial task that edits files and runs commands. Users reported vLLM rejecting Vibe's tool-call requests, the Devstral 2 template on llama.cpp failing after interrupts, and llama.cpp keep-alive events breaking streaming in 2.14; check the release notes for each fix.
Budget context
How much context does a real task need?
Look forRoom for Vibe's system prompt and tool definitions on top of the code. One user on a small context ran out at about 16k tokens, and one on a 92 GB MacBook kept sessions under about 80k tokens to stay stable.
Turn off what calls out
Is the setup offline, or only the model?
Look forenable_telemetry and enable_update_checks set to false, and skills, MCP servers, and connectors that reach external systems disabled, as Mistral's offline guide lists.
Questions to ask vendors
- Which open-weight models does Mistral test Vibe's tool prompts against, and on which servers?
- Will Mistral publish fixed chat templates for Devstral models on llama.cpp and vLLM?
- Does admin config apply to a machine that runs fully offline?
- Which Vibe features need the hosted service even with a local model?
Red flags
- A local model reaches the team without tests of its edit and shell tool calls.
- Telemetry and update checks stay on in an environment described as offline.
- Context is set at the server default with no check of what a real task needs.
- MCP servers or connectors that reach the internet stay enabled on an offline machine.
Products in this piece
Sources
- Official documentation
- 6
- Public source
- 7
- Mistral docs: Using offline models
- Mistral docs: Configuration
- Mistral: Introducing Devstral 2 and Mistral Vibe CLI (Dec 9, 2025)
- Mistral docs: Install and setup
- Mistral docs: MCP servers
- Mistral Vibe releases on GitHub
- Reddit r/LocalLLaMA: Mistral Vibe and Devstral 2 Small as a local combo (Jan 4, 2026)
- Reddit r/LocalLLaMA: Mistral Vibe CLI with Qwen 4B Q4 (Dec 14, 2025)
- Reddit r/LocalLLaMA: Mistral Vibe with Devstral Small on a 92 GB MacBook (Feb 2, 2026)
- Hacker News, asimovDev: a local model in Vibe running out of context at about 16k (Mar 2, 2026)
- GitHub issue #70: tool calls fail against a local vLLM Devstral 2 backend (Dec 11, 2025)
- GitHub issue #255: interrupts break the Devstral 2 chat template on llama.cpp (Jan 16, 2026)
- GitHub issue #762: llama.cpp keep-alive events break streaming (Jun 10, 2026)