Cost
$1.61per task
Cost per task. One Intelligence Index task at list price, every token counted. Run at mistral Large 4 Preview.
- Cached input
- $0.07 per 1M tokens
Mistral Large model by Mistral AI
Mistral Large 4
Mistral AI's flagship multimodal mixture-of-experts model, in Public Preview (v26.10) on the Mistral API and Mistral Studio since October 6, 2026, and also on OpenRouter and OpenCode Zen. Mistral has published no GA date. Its open weights followed on October 29, 2026, under the Mistral Ecosystem Open-Weight license, a modified MIT license.
Where it sits in Mistral AI's lineup
- Mistral LargeMistral AI's flagship general-purpose model family.This model
- Mistral MediumMistral AI's mid-size model family for agentic and coding work.Mistral Medium 3.5
- Stack role
- Foundation Model / Reasoning Engine
- Offered in
- 1 product
- Made by
- Mistral AI
- Family
- Mistral Large, version 4
- Released
- Oct 6, 2026 Current
- Lifecycle
- Public preview, per Mistral AI.
- In its family
- The only Mistral Large release we track.
Strengths and weaknesses
Compare with other models Up to 3 beside Mistral Large 4
- Mistral Large 4
Suggested comparisons
The models a buyer weighs against Mistral Large 4, picked on list price and on Artificial Analysis's Intelligence Index of overall quality. Add one to draw it beside Mistral Large 4.
Mistral Large 4
Among the 40 current releases on this chart, Mistral Large 4's best rank is 24th of 38 on Tool use. Agentic coding is Artificial Analysis's run on Terminal-Bench 4.0, marked on the chart. It ranks lowest on Factuality, 28th of 30. Computer use and Instruction following: not yet published by Mistral AI or any independent evaluator, checked Nov 2, 2026. Code, Math, Science, Hard reasoning, Long context, Multimodal and Answers in full: published only for another harness, setting or build, listed in the table.
Each axis is one public benchmark at one setting, on its own fixed 0 to 100 scale, so adding a model never moves another. A dotted bridge crosses a missing result, which is never a zero. A hollow mark is a result from another source: the maker's own figure or another lab's run, where the benchmark's usual source has not tested the model yet. The readout names it.
On the chart
- Mistral Large 4This pageResults on 3 of 12 axes$0.70 / $2.10161 tokens/sFirst token in 1.4 s
Prices and speeds as of Oct 30, 2026.
Mistral Large 4 on each axis3 of 12 with a result
The share of Terminal-Bench 4.0's long terminal tasks the model finishes on its own: services, kernels, proofs and data pipelines, graded on the final artifact with no partial credit.
Terminal-Bench 4.0, pass@1 averaged over 3 runs, mini-swe-agent harness. Where it has no run of a model, the maker's own figure or another lab's run stands in, marked with its benchmark.
- Mistral Large 424.7%Artificial Analysis's runBenchmark: Terminal-Bench 4.0, accuracy, Artificial Analysis run in its own agent harness, where the axis reads pass@1 averaged over 3 runs, mini-swe-agent harness. From another source, so it is drawn and marked but not ranked with the others.Artificial Analysis's run of the October 29 build at high effort in its own agent harness. Vals.ai has run only the October 6 preview build.
The share of SciCode's scientific programming subproblems the model's code solves on the first attempt.
SciCode, subproblem accuracy, pass@1, highest effort run
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Artificial Analysis 54.2% on an earlier build. The table below lists it.
The share of FrontierMath's held-out research-grade problems (tiers 1 to 3) the model answers correctly.
FrontierMath Tiers 1-3, accuracy, pass@1, highest effort run
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Epoch AI 58.6% on an earlier build. The table below lists it.
The share of GPQA Diamond's graduate-level biology, chemistry and physics questions the model answers correctly.
GPQA Diamond, accuracy, pass@1, highest effort run
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Epoch AI 88.1% on an earlier build. The table below lists it.
The share of Humanity's Last Exam's 2,158 text-only questions, expert-written at the edge of human knowledge, the model answers correctly without tools.
Humanity's Last Exam, accuracy, text-only questions, no tools
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Artificial Analysis 35% on an earlier build. The table below lists it.
The share of each AutomationBench task's objectives the model completes by calling REST APIs across simulated business apps (Gmail, Sheets, Slack, Salesforce, Zendesk, Jira, HubSpot), averaged over tasks. A task that breaks one of its policy guardrails scores zero.
AutomationBench-AA, score, Artificial Analysis run
- Mistral Large 457.8%Artificial Analysis's run of the October 29 build at high effort, its highest run of that build.
How much of each OSWorld 2.1 workflow the model completes by operating a real Ubuntu desktop through screenshots, mouse and keyboard: all 108 long tasks across apps, files, mail and browsers, on the September 2026 v2.1 release of the task files. A point is the mean share of each task's weighted checks the model passes (the partial score, as a percentage). The OSWorld team's own runs come first. Where the team has not run a release, the point is another lab's run or the maker's own published figure, marked with who ran it and the benchmark it ran: OSWorld 2.1 in the maker's harness, OSWorld 2.0's offline set or the earlier OSWorld-Verified. Those points are drawn so the release has a figure, and they are not ranked against the team's runs.
OSWorld 2.1, partial score, full 108-task set, v2.1 release
- Mistral Large 4Not yet published by Mistral AI or any independent evaluator, checked Nov 2, 2026
The share of AA-LCR's questions the model answers correctly by reasoning across long sets of real documents, about 100K tokens each.
AA-LCR v1.1, accuracy, Artificial Analysis run
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Artificial Analysis 81.3% on an earlier build. The table below lists it.
The share of MMMU-Pro's college-level questions over charts, diagrams and photos the model answers correctly.
MMMU-Pro, overall accuracy, no tools
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: Artificial Analysis 76.4% on an earlier build. The table below lists it.
The share of AA-Omniscience's 6,000 knowledge questions across 42 economically relevant topics the model answers correctly from memory, without search.
AA-Omniscience Accuracy, accuracy, Artificial Analysis run
- Mistral Large 426%Artificial Analysis's run of the October 29 build at high effort, its highest run of that build.Hallucination rate 28.2% in the same run (lower is better)
The share of SpeechMap's 2,120 sensitive and controversial but legal requests the model answers in full: arguments, essays and explanations on contested themes in politics, religion, race, sex, history and similar topics, four wordings per theme across 530 themes. SpeechMap's judge rates each response complete, evasive (hedged or redirected), a denial, or an error (the provider's API blocked the request). Only complete answers count, so the point is the share answered in full: 100 minus the share hedged, refused or blocked by the provider. Higher means more lawful requests answered in full. The share blocked at the provider is printed apart in the table, because an API error is not the model declining. Where SpeechMap runs a release both with reasoning off and with reasoning on (a -reasoning or -thinking entry), the reasoning-on entry is used; a release with one entry uses that entry.
SpeechMap, % Complete over 2,120 contested requests, reasoning-on entry
- Mistral Large 4Published only for another harness, setting or build, listed in the tableNot drawn: SpeechMap.AI 48.5% on an earlier build. The table below lists it.
The share of MultiChallenge's multi-turn conversations in which the model keeps every instruction it was given earlier in the conversation.
MultiChallenge, accuracy, Scale AI run
- Mistral Large 4Not yet published by Mistral AI or any independent evaluator, checked Nov 2, 2026
Cost, speed, capacity, quality and accuracy for Mistral Large 4, with where each figure sits among the 40 models GILD tracks. These measures stand beside the chart and never feed it.
Speed
116tokens a second
Output speed. How fast it writes once the answer starts. Run at mistral Large 4 Preview.
- First token
- 1.6 s
- Whole answer
- 23 s
Capacity
1Mtokens
Context window. The most one request can hold, as the maker states it.
- Max output
- 262K
- Reads
- Text, Images
Quality
38.4of 100
Intelligence Index. One composite score across independent evaluations. Run at mistral Large 4 Preview.
- GDPval knowledge work
- 1,434 Elo
- Terminal-Bench 4.0
- 26.8%
- Epoch Capabilities Index
- 150
Hallucination
28.2%wrong when it should say “I don’t know”
Hallucination rate. Of the questions it can't get right, how often it answers anyway. Lower is better. Run at high effort.
- Knowledge accuracy
- 26.0%
Where you can get it
3of 19 routes
Routes offering it. Of the clouds, gateways and coding products GILD tracks, those offering it today.
- Open weights
- Yes
- Licence
- Mistral Ecosystem Open Weight License
Figures from Artificial Analysis, Mistral AI, OpenRouter, Epoch AI and GILD's route matrix, read Oct 26 to 30, 2026. Each measure's definition and source open under .
Every value with its benchmark, run and source
| Axis | Benchmark and setting | Mistral Large 4 | Run | Source | Date |
|---|---|---|---|---|---|
| Agentic coding | Terminal-Bench 4.0Accuracy, Artificial Analysis run in its own agent harness. Artificial Analysis's run of the October 29 build at high effort in its own agent harness. Vals.ai has run only the October 6 preview build.Stands in for Terminal-Bench 4.0, pass@1 averaged over 3 runs, mini-swe-agent harness, which has no run of Mistral Large 4 yet. | 24.7% | Mistral Large 4 (High)Effort: high. Harness: Artificial Analysis agent harness. Run on the Oct 29, 2026 build.Not drawn: Vals.ai 27.8% on an earlier build, Oct 24, 2026. Source | Artificial AnalysisRun by Artificial Analysis | Oct 30, 2026 |
| Code | SciCodeSubproblem accuracy, pass@1, highest effort run | Published only for another harness, setting or build, listed in the table | Not drawn: Artificial Analysis 54.2% on an earlier build, Oct 27, 2026. Source | ||
| Math | FrontierMath Tiers 1-3Accuracy, pass@1, highest effort run | Published only for another harness, setting or build, listed in the table | Not drawn: Epoch AI 58.6% on an earlier build, Oct 12, 2026. Source | ||
| Science | GPQA DiamondAccuracy, pass@1, highest effort run | Published only for another harness, setting or build, listed in the table | Not drawn: Epoch AI 88.1% on an earlier build, Oct 12, 2026. Source | ||
| Hard reasoning | Humanity's Last ExamAccuracy, text-only questions, no tools | Published only for another harness, setting or build, listed in the table | Not drawn: Artificial Analysis 35% on an earlier build, Oct 27, 2026. Source | ||
| Tool use | AutomationBench-AAScore, Artificial Analysis run. Artificial Analysis's run of the October 29 build at high effort, its highest run of that build. | 57.8% | Mistral Large 4 (High)Effort: high. Harness: Artificial Analysis AutomationBench harness. Run on the Oct 29, 2026 build.Not drawn: Artificial Analysis 59.9% on an earlier build, Oct 27, 2026. Source | Artificial Analysis | Oct 30, 2026 |
| Computer use | OSWorld 2.1Partial score, full 108-task set, v2.1 release | Not yet published by Mistral AI or any independent evaluator, checked Nov 2, 2026 | |||
| Long context | AA-LCR v1.1Accuracy, Artificial Analysis run | Published only for another harness, setting or build, listed in the table | Not drawn: Artificial Analysis 81.3% on an earlier build, Oct 27, 2026. Source | ||
| Multimodal | MMMU-ProOverall accuracy, no tools | Published only for another harness, setting or build, listed in the table | Not drawn: Artificial Analysis 76.4% on an earlier build, Oct 27, 2026. Source | ||
| Factuality | AA-Omniscience AccuracyAccuracy, Artificial Analysis run. Artificial Analysis's run of the October 29 build at high effort, its highest run of that build. | 26%Hallucination rate 28.2% in the same run (lower is better) | Mistral Large 4 (High)Effort: high. Harness: none. Run on the Oct 29, 2026 build.Not drawn: Artificial Analysis 23.9% on an earlier build, Oct 28, 2026. Source | Artificial Analysis | Oct 30, 2026 |
| Answers in full | SpeechMap% Complete over 2,120 contested requests, reasoning-on entry | Published only for another harness, setting or build, listed in the table | Not drawn: SpeechMap.AI 48.5% on an earlier build, Oct 27, 2026. Source | ||
| Instruction following | MultiChallengeAccuracy, Scale AI run | Not yet published by Mistral AI or any independent evaluator, checked Nov 2, 2026 |
Price and speed
- Mistral Large 4 lists at $0.70 per million input tokens and $2.10 per million output tokens. Mistral AI pricing, Oct 30, 2026.
- Mistral Large 4, at high effort, writes 161.3 output tokens per second on Mistral's API, and its first token arrives 1.4 s after the request. Artificial Analysis, Oct 30, 2026.
- Fastest on the chart is the shortest time to a 500-token answer: time to first token plus 500 tokens at the median output speed.
Every model with a result, axis by axis
57 models. A maker's own figure and another lab's run are marked with their benchmark; a missing result reads "Not yet published", never zero.
| Model | Agentic coding | Code | Math | Science | Hard reasoning | Tool use | Computer use | Long context | Multimodal | Factuality | Answers in full | Instructions |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mistral Large 4 | 24.7%Run by Artificial AnalysisTerminal-Bench 4.0 | Not yet published | Not yet published | Not yet published | Not yet published | 57.8% | Not yet published | Not yet published | Not yet published | 26% | Not yet published | Not yet published |
| ||||||||||||
| Mistral Medium 3.5 | 0%Run by Artificial AnalysisTerminal-Bench 4.0 | 40.2% | Not yet published | 74.8%Run by Artificial AnalysisGPQA Diamond | 13.8% | 6.3% | Not yet published | 69.3% | 64.9% | 23% | 70.5% | 68.8%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Claude Haiku 5.5 | 36.4% | 55% | 75.1% | 89.6% | 44.4% | 35.4% | 72.4%Reported by AnthropicOSWorld 2.1 | 82.7% | Not yet published | 34.7% | 78.8% | 67.7% |
| ||||||||||||
| Claude Sonnet 5.5 | 64.1% | 61% | 88.8% | 95.6% | 55% | 71.8% | 80.1%Reported by Anthropic | 82.7% | Not yet published | 51.5% | 81.6% | Not yet published |
| ||||||||||||
| Claude Opus 5.5 | 65.2% | 66.9% | 91.2% | 90.6% | 61.4% | 69.5% | 81.8%Reported by Anthropic | 84.7% | 87.7% | 65.5% | 82.8% | Not yet published |
| ||||||||||||
| Claude Fable 5.1 | 58.1% | 63.1% | 90.2% | 93.4%Run by Vals.ai | 59.1% | 59.4% | 80.7%Reported by Anthropic | 85.3% | 86.3%Run by Moonshot AI | 68.4% | 72.4% | Not yet published |
| ||||||||||||
| Claude Opus 5 | 53.5% | 56.4% | 85.6% | 93.9% | 54.9% | 56.6% | 77.7% | 79.3% | 84.7% | 60.9% | 73.7% | Not yet published |
| ||||||||||||
| Claude Sonnet 5 | 10.6% | 54.3% | 65.6% | 80.3% | 41.3% | 36.5% | 57%Reported by Anthropic | 82% | 77.3% | 40.1% | Not yet published | Not yet published |
| ||||||||||||
| Claude Fable 5 | 41.4% | 61% | 87% | 85.9% | 55.5% | 54.1% | Not yet published | 82.3% | Not yet published | 65.4% | 64.8% | 63.5%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Claude Opus 4.8 | 23.2% | 54.4% | 80% | 91% | 48.7% | 45.6% | 54.8%Run by OSWorld teamOSWorld 2.0 | 77.7% | Not yet published | 48.8% | 67.4% | 62.2%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Claude Opus 4.7 | Not yet published | 54.5% | 70.2% | 86.4% | 42.3% | Not yet published | 48.9%Run by OSWorld teamOSWorld 2.0 | 78.7% | 78.8% | 48.9% | 70.2% | 58.6%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Claude Sonnet 4.6 | 3%Run by Artificial AnalysisTerminal-Bench 4.0 | 50.1% | 71.1%Run by Epoch AIOTIS Mock AIME 2024-2025 | 78.8% | 33.6% | 20.1% | 41.5%Run by OSWorld teamOSWorld 2.0 | 80% | 73.3% | 40.9% | 35.8% | 56.6%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Claude Opus 4.6 | Not yet published | Not yet published | 66% | 88.4% | 39.9% | Not yet published | Not yet published | 78% | 75.4% | 47% | 59.9% | 37.2% |
| ||||||||||||
| Claude Opus 4.5 | Not yet published | Not yet published | 34.4% | 86% | 30.1% | Not yet published | Not yet published | 77.3% | 74% | 46.6% | 57.5% | 59% |
| ||||||||||||
| Claude Haiku 4.5 | Not yet published | 42.2% | 66.7%Run by Epoch AIOTIS Mock AIME 2024-2025 | 71.2% | 10.4% | 3.2% | Not yet published | 74.3% | 58.6% | 18% | 24.7% | 50.5% |
| ||||||||||||
| Claude Sonnet 4.5 | Not yet published | 45.7% | 23.9% | 82.3% | 17.8% | 14.1% | Not yet published | 72.3% | 68.7% | 32.9% | 31.4% | 55.3% |
| ||||||||||||
| GPT-6.1 Terra | 49.5% | 55.1% | 90.9% | 94.3% | 46.2% | 60.2% | 74.2%Reported by OpenAIOSWorld 2.1 | 82.7% | 84.7% | 47.6% | Not yet published | 74.2% |
| ||||||||||||
| GPT-6.1 Sol | 55.1% | 54.2% | 93.7% | 95.4% | 52.9% | 64.9% | 78%Reported by OpenAIOSWorld 2.1 | 83% | 86% | 61.8% | 69.5% | 82% |
| ||||||||||||
| GPT-6 Luna | 13.6% | 54.6% | 78.9% | 90.5% | 38.5% | 53.2% | 48.9%Run by AnthropicOSWorld 2.1 | 83.3% | 79.7% | 39.6% | 63.4% | Not yet published |
| ||||||||||||
| GPT-6 Astra | 59.6% | 56.5% | 93.7% | 95.8% | 54.7% | 68.5% | 75.8%Run by Meta | 80.7% | 86.9% | 61.6% | 68.2% | Not yet published |
| ||||||||||||
| GPT-5.6 Sol | 37.9% | 57.1% | 89.1% | 93.5% | 49.5% | 60.1% | 62.7%Run by OSWorld teamOSWorld 2.0 | 84% | 83.4% | 69.7%Run by Epoch AISimpleQA Verified | Not yet published | 72.7%Run by Artificial AnalysisIFBench |
| ||||||||||||
| GPT-5.6 Terra | 22.7% | 55% | 86% | 93.3% | 42.9% | 59.6% | Not yet published | 83% | 80.7% | 42.3% | Not yet published | 71.2%Run by Artificial AnalysisIFBench |
| ||||||||||||
| GPT-5.6 Luna | 11.6% | 53.6% | 82.1% | 91.6% | 39.5% | 50.2% | Not yet published | 83.7% | 78.6% | 41%Run by Epoch AISimpleQA Verified | Not yet published | Not yet published |
| ||||||||||||
| GPT-5.5 | 14.6%Run by Artificial AnalysisTerminal-Bench 4.0 | 55.8% | 85.3% | 93.5%Run by Artificial Analysis | 45.8% | 47.3% | 49.5%Run by OSWorld teamOSWorld 2.0 | 84.3% | 79.9% | 63%Run by Epoch AISimpleQA Verified | Not yet published | 75.9%Run by Artificial AnalysisIFBench |
| ||||||||||||
| GPT-5.4 | Not yet published | Not yet published | 78.6% | 93.3% | 43.7% | Not yet published | Not yet published | 82% | 78.4% | Not yet published | Not yet published | 73.9%Run by Artificial AnalysisIFBench |
| ||||||||||||
| gpt-oss-120b | 14.2%Run by Terminal-Bench (via Epoch AI)Terminal-Bench 2.0 | 34% | 88.9%Run by Epoch AIOTIS Mock AIME 2024-2025 | 75.8% | 19.6% | 0.2% | Not yet published | 52% | Not yet published | 16.8%Reported by OpenAISimpleQA | 36.3% | 45.3% |
| ||||||||||||
| GPT-6 Sol | 44.4% | 57.6% | 89.8% | 94.3% | 47.9% | 61.6% | 64.4%Reported by OpenAIOSWorld 2.0 | 83.7% | 82.9% | 54.5% | 64% | Not yet published |
| ||||||||||||
| GPT-5.3-Codex | Not yet published | Not yet published | Not yet published | 91.5%Run by Artificial Analysis | 42.5% | Not yet published | Not yet published | 83.3% | 78.5% | Not yet published | Not yet published | 75.4%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Gemini 4 Argon | 57.6% | 61.8% | 86% | 96.4% | 57.1% | 77.5% | 69.2%Reported by GoogleOSWorld 2.0 | 79.7% | Not yet published | 48.3% | 87% | 78.7% |
| ||||||||||||
| Gemini 3.8 Flash | 19.7% | 56.6% | 68.4% | 95.4% | 47.8% | 59.9% | 59%Reported by GoogleOSWorld 2.0 | 81.3% | 85.6% | 50.5% | 86.1% | Not yet published |
| ||||||||||||
| Gemma 4 31B | Not yet published | 45.5% | Not yet published | 85.7%Run by Artificial AnalysisGPQA Diamond | 23.6% | 4.9% | Not yet published | 69.7% | 73.4% | 10.4%Run by Epoch AISimpleQA Verified | 82.3% | 75.6%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Gemini 3.1 Pro Preview | 2.5% | 58.7% | 59.6% | 94.4% | 47% | 35.4% | Not yet published | 82% | 82.4% | 54.9% | 59.9% | 71.4% |
| ||||||||||||
| Gemini 3.6 Flash | 7.1%Run by Artificial AnalysisTerminal-Bench 4.0 | 53.4% | 58.9% | 94.1% | 40.8% | 53% | 83%Reported by GoogleOSWorld-Verified | 80% | 83.2% | 66.2%Run by Epoch AISimpleQA Verified | 87.2% | Not yet published |
| ||||||||||||
| DeepSeek V4.1 Flash | 31.8% | 55.9% | 71.2% | 93.2% | 43.3% | 66.6% | 69.3%Run by Moonshot AIOSWorld 2.1 | 82.7% | 80.6% | 39.6% | Not yet published | 74.9% |
| ||||||||||||
| DeepSeek V4 Pro | 14.1% | 51% | 64.6% | 91.7% | 41% | 56.7% | Not yet published | 80.3% | Not yet published | 45.2% | 72% | Not yet published |
| ||||||||||||
| Kimi K3.1 Pro | 63.6% | 60.8% | Not yet published | 94.2%Reported by Moonshot AI | 52.5% | 73.3% | 79.6%Reported by Moonshot AIOSWorld 2.1 | 86.3% | 83.9% | 48.9% | Not yet published | Not yet published |
| ||||||||||||
| Kimi K3 | 17.2% | 59.5% | 72.2% | 93.1% | 46.9% | 58.3% | Not yet published | 88.7% | 80.5% | 50.6%Run by Epoch AISimpleQA Verified | 65.9% | Not yet published |
| ||||||||||||
| Kimi K2.7 Code | 1%Run by Artificial AnalysisTerminal-Bench 4.0 | 47.8% | 54% | 87.9% | 35% | 24.5% | Not yet published | 79.3% | Not yet published | 36.5%Run by Epoch AISimpleQA Verified | Not yet published | 63.1%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Grok 4.7 | 28.8% | 57.4% | 53% | 92.7% | 43.1% | 65.6% | Not yet published | 76.7% | Not yet published | 42.7% | 64.1% | Not yet published |
| ||||||||||||
| Grok 4.6 | 17.2% | 56.5% | 66% | 93.2% | 42.9% | 66.7% | Not yet published | 80.3% | Not yet published | 48.9%Run by Epoch AISimpleQA Verified | 83.6% | Not yet published |
| ||||||||||||
| Grok 4.5 | 8.6% | 55% | 57.2% | 93.4% | 42.7% | 57.9% | Not yet published | 79.3% | 80.4% | 48.3%Run by Epoch AISimpleQA Verified | 93.1% | Not yet published |
| ||||||||||||
| MiniMax M3.1 Flash | 25.3% | 53.4% | Not yet published | Not yet published | 48% | 72.5% | 61.7%Reported by MiniMaxOSWorld 2.0 | 79.7% | 79.1% | 45.1% | Not yet published | Not yet published |
| ||||||||||||
| Mercury 3 | 3% | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published | Not yet published |
| GLM-5.3-Flash | 37.4% | 51.6% | 55.8% | 90.2% | 39.9% | 60.4% | 59.1%Reported by Z.aiOSWorld 2.0 | 80% | 86%Run by Vals.aiMMMU-Pro | 28.2% | 65.4% | Not yet published |
| ||||||||||||
| GLM-5.3 | 38.9% | 59% | 68.8% | 90.9% | 42.3% | 62.2% | Not yet published | 79.7% | Not yet published | 32.3% | 71.5% | Not yet published |
| ||||||||||||
| GLM-5.2 | 1%Run by Artificial AnalysisTerminal-Bench 4.0 | 51.2% | 59.2% | Not yet published | 41.1% | 28.4% | Not yet published | 78.3% | Not yet published | 34.2%Run by Epoch AISimpleQA Verified | Not yet published | 73.3%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Muse Spark Open | 6.6% | 58.4% | 69.8% | 88.6% | 43.9% | 49.7% | 80.4%Reported by MetaOSWorld-Verified | 78.3% | 80.9% | 42% | Not yet published | 74.2% |
| ||||||||||||
| Muse Wonder | 49.5% | 60.3% | 82.1% | Not yet published | 50.1% | 68.1% | 72.6%Reported by Meta | 80.7% | Not yet published | 38% | 42.7% | Not yet published |
| ||||||||||||
| Muse Spark 1.3 | 10.6% | 58.8% | 74% | 93.5%Run by Artificial AnalysisGPQA Diamond | 48.7% | 57.9% | 69.6% | 83% | 82% | 38% | 25.8% | Not yet published |
| ||||||||||||
| Llama 4 Scout | Not yet published | 21.3% | 7.8%Run by Epoch AIOTIS Mock AIME 2024-2025 | 51.8% | 3.8% | 0.2% | Not yet published | 27.7% | 52.9% | Not yet published | 70.1% | 39.5%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Llama 4 Maverick | Not yet published | 31.7% | 20.6%Run by Epoch AIOTIS Mock AIME 2024-2025 | 67% | 4.9% | 0.2% | Not yet published | 50% | 62.1% | Not yet published | 58.3% | 43%Run by Artificial AnalysisIFBench |
| ||||||||||||
| Qwen3.8 27B | 5.6%Run by Artificial AnalysisTerminal-Bench 4.0 | 46.6% | Not yet published | 90.5%Run by Artificial AnalysisGPQA Diamond | 33.9% | 48.2% | 84.3%Reported by AlibabaOSWorld-Verified | 82% | 76.3% | 17.3% | Not yet published | 79.5%Reported by AlibabaIFBench |
| ||||||||||||
| Qwen3.8 Max | 38.9% | 52.1% | 65.6% | 92.3% | 43.1% | 56.2% | Not yet published | 80.3% | 82.8% | 29.5% | 65.3% | Not yet published |
| ||||||||||||
| MiMo-V2.6-Pro | 31.3% | 60.9% | Not yet published | Not yet published | 49.4% | 58.6% | 82%Reported by XiaomiOSWorld-Verified | 86.3% | Not yet published | 34.8% | 66.5% | Not yet published |
| ||||||||||||
| MiMo-V2.6-Flash | 24.2% | 51.3% | Not yet published | Not yet published | 35.1% | 64.1% | 80.8%Reported by XiaomiOSWorld-Verified | 74.3% | 73.1% | 27.2% | 66.9% | Not yet published |
| ||||||||||||
| Ling 3.1 Flash | 32.8%Run by Artificial AnalysisTerminal-Bench 4.0 | 56.8% | Not yet published | Not yet published | 43.5% | 67.5% | Not yet published | 82.3% | Not yet published | 26.5% | Not yet published | Not yet published |
| ||||||||||||
| Step 5 Preview | 31.8% | 58.9% | Not yet published | 93.5%Reported by StepFunGPQA Diamond | 46.5% | 51% | Not yet published | 88.3% | 76.4% | 36.4% | 59.7% | Not yet published |
| ||||||||||||
Agentic codingTerminal-Bench 4.0
- Claude Opus 5.565.2%
- Claude Sonnet 5.564.1%
- Kimi K3.1 Pro63.6%
- GPT-6 Astra59.6%
- Claude Fable 5.158.1%
- Gemini 4 Argon57.6%
- GPT-6.1 Sol55.1%
- Claude Opus 553.5%
- GPT-6.1 Terra49.5%
- Muse Wonder49.5%
- GPT-6 Sol44.4%
- Claude Fable 541.4%
- GLM-5.338.9%
- Qwen3.8 Max38.9%
- GPT-5.6 Sol37.9%
- GLM-5.3-Flash37.4%
- Claude Haiku 5.536.4%
- DeepSeek V4.1 Flash31.8%
- Step 5 Preview31.8%
- MiMo-V2.6-Pro31.3%
- Grok 4.728.8%
- MiniMax M3.1 Flash25.3%
- MiMo-V2.6-Flash24.2%
- Claude Opus 4.823.2%
- GPT-5.6 Terra22.7%
- Gemini 3.8 Flash19.7%
- Kimi K317.2%
- Grok 4.617.2%
- DeepSeek V4 Pro14.1%
- GPT-6 Luna13.6%
- GPT-5.6 Luna11.6%
- Claude Sonnet 510.6%
- Muse Spark 1.310.6%
- Grok 4.58.6%
- Muse Spark Open6.6%
- Mercury 33%
- Gemini 3.1 Pro Preview2.5%
- Ling 3.1 Flash32.8%Artificial Analysis's run, Terminal-Bench 4.0
- Mistral Large 424.7%Artificial Analysis's run, Terminal-Bench 4.0
- GPT-5.514.6%Artificial Analysis's run, Terminal-Bench 4.0
- gpt-oss-120b14.2%Terminal-Bench's run, Terminal-Bench 2.0
- Gemini 3.6 Flash7.1%Artificial Analysis's run, Terminal-Bench 4.0
- Qwen3.8 27B5.6%Artificial Analysis's run, Terminal-Bench 4.0
- Claude Sonnet 4.63%Artificial Analysis's run, Terminal-Bench 4.0
- Kimi K2.7 Code1%Artificial Analysis's run, Terminal-Bench 4.0
- GLM-5.21%Artificial Analysis's run, Terminal-Bench 4.0
- Mistral Medium 3.50%Artificial Analysis's run, Terminal-Bench 4.0
The other models in the grid have no result on Terminal-Bench 4.0 yet.
CodeSciCode
- Claude Opus 5.566.9%
- Claude Fable 5.163.1%
- Gemini 4 Argon61.8%
- Claude Sonnet 5.561%
- Claude Fable 561%
- MiMo-V2.6-Pro60.9%
- Kimi K3.1 Pro60.8%
- Muse Wonder60.3%
- Kimi K359.5%
- GLM-5.359%
- Step 5 Preview58.9%
- Muse Spark 1.358.8%
- Gemini 3.1 Pro Preview58.7%
- Muse Spark Open58.4%
- GPT-6 Sol57.6%
- Grok 4.757.4%
- GPT-5.6 Sol57.1%
- Ling 3.1 Flash56.8%
- Gemini 3.8 Flash56.6%
- GPT-6 Astra56.5%
- Grok 4.656.5%
- Claude Opus 556.4%
- DeepSeek V4.1 Flash55.9%
- GPT-5.555.8%
- GPT-6.1 Terra55.1%
- Claude Haiku 5.555%
- GPT-5.6 Terra55%
- Grok 4.555%
- GPT-6 Luna54.6%
- Claude Opus 4.754.5%
- Claude Opus 4.854.4%
- Claude Sonnet 554.3%
- GPT-6.1 Sol54.2%
- GPT-5.6 Luna53.6%
- Gemini 3.6 Flash53.4%
- MiniMax M3.1 Flash53.4%
- Qwen3.8 Max52.1%
- GLM-5.3-Flash51.6%
- MiMo-V2.6-Flash51.3%
- GLM-5.251.2%
- DeepSeek V4 Pro51%
- Claude Sonnet 4.650.1%
- Kimi K2.7 Code47.8%
- Qwen3.8 27B46.6%
- Claude Sonnet 4.545.7%
- Gemma 4 31B45.5%
- Claude Haiku 4.542.2%
- Mistral Medium 3.540.2%
- gpt-oss-120b34%
- Llama 4 Maverick31.7%
- Llama 4 Scout21.3%
The other models in the grid have no result on SciCode yet.
MathFrontierMath Tiers 1-3
- GPT-6.1 Sol93.7%
- GPT-6 Astra93.7%
- Claude Opus 5.591.2%
- GPT-6.1 Terra90.9%
- Claude Fable 5.190.2%
- GPT-6 Sol89.8%
- GPT-5.6 Sol89.1%
- Claude Sonnet 5.588.8%
- Claude Fable 587%
- GPT-5.6 Terra86%
- Gemini 4 Argon86%
- Claude Opus 585.6%
- GPT-5.585.3%
- GPT-5.6 Luna82.1%
- Muse Wonder82.1%
- Claude Opus 4.880%
- GPT-6 Luna78.9%
- GPT-5.478.6%
- Claude Haiku 5.575.1%
- Muse Spark 1.374%
- Kimi K372.2%
- DeepSeek V4.1 Flash71.2%
- Claude Opus 4.770.2%
- Muse Spark Open69.8%
- GLM-5.368.8%
- Gemini 3.8 Flash68.4%
- Claude Opus 4.666%
- Grok 4.666%
- Claude Sonnet 565.6%
- Qwen3.8 Max65.6%
- DeepSeek V4 Pro64.6%
- Gemini 3.1 Pro Preview59.6%
- GLM-5.259.2%
- Gemini 3.6 Flash58.9%
- Grok 4.557.2%
- GLM-5.3-Flash55.8%
- Kimi K2.7 Code54%
- Grok 4.753%
- Claude Opus 4.534.4%
- Claude Sonnet 4.523.9%
- gpt-oss-120b88.9%Epoch AI's run, OTIS Mock AIME 2024-2025
- Claude Sonnet 4.671.1%Epoch AI's run, OTIS Mock AIME 2024-2025
- Claude Haiku 4.566.7%Epoch AI's run, OTIS Mock AIME 2024-2025
- Llama 4 Maverick20.6%Epoch AI's run, OTIS Mock AIME 2024-2025
- Llama 4 Scout7.8%Epoch AI's run, OTIS Mock AIME 2024-2025
The other models in the grid have no result on FrontierMath Tiers 1-3 yet.
ScienceGPQA Diamond
- Gemini 4 Argon96.4%
- GPT-6 Astra95.8%
- Claude Sonnet 5.595.6%
- GPT-6.1 Sol95.4%
- Gemini 3.8 Flash95.4%
- Gemini 3.1 Pro Preview94.4%
- GPT-6.1 Terra94.3%
- GPT-6 Sol94.3%
- Gemini 3.6 Flash94.1%
- Claude Opus 593.9%
- GPT-5.6 Sol93.5%
- Grok 4.593.4%
- GPT-5.6 Terra93.3%
- GPT-5.493.3%
- DeepSeek V4.1 Flash93.2%
- Grok 4.693.2%
- Kimi K393.1%
- Grok 4.792.7%
- Qwen3.8 Max92.3%
- DeepSeek V4 Pro91.7%
- GPT-5.6 Luna91.6%
- Claude Opus 4.891%
- GLM-5.390.9%
- Claude Opus 5.590.6%
- GPT-6 Luna90.5%
- GLM-5.3-Flash90.2%
- Claude Haiku 5.589.6%
- Muse Spark Open88.6%
- Claude Opus 4.688.4%
- Kimi K2.7 Code87.9%
- Claude Opus 4.786.4%
- Claude Opus 4.586%
- Claude Fable 585.9%
- Claude Sonnet 4.582.3%
- Claude Sonnet 580.3%
- Claude Sonnet 4.678.8%
- gpt-oss-120b75.8%
- Claude Haiku 4.571.2%
- Llama 4 Maverick67%
- Llama 4 Scout51.8%
- Kimi K3.1 Pro94.2%Moonshot AI's own figure
- GPT-5.593.5%Artificial Analysis's run
- Muse Spark 1.393.5%Artificial Analysis's run, GPQA Diamond
- Step 5 Preview93.5%StepFun's own figure, GPQA Diamond
- Claude Fable 5.193.4%Vals.ai's run
- GPT-5.3-Codex91.5%Artificial Analysis's run
- Qwen3.8 27B90.5%Artificial Analysis's run, GPQA Diamond
- Gemma 4 31B85.7%Artificial Analysis's run, GPQA Diamond
- Mistral Medium 3.574.8%Artificial Analysis's run, GPQA Diamond
The other models in the grid have no result on GPQA Diamond yet.
Hard reasoningHumanity's Last Exam
- Claude Opus 5.561.4%
- Claude Fable 5.159.1%
- Gemini 4 Argon57.1%
- Claude Fable 555.5%
- Claude Sonnet 5.555%
- Claude Opus 554.9%
- GPT-6 Astra54.7%
- GPT-6.1 Sol52.9%
- Kimi K3.1 Pro52.5%
- Muse Wonder50.1%
- GPT-5.6 Sol49.5%
- MiMo-V2.6-Pro49.4%
- Claude Opus 4.848.7%
- Muse Spark 1.348.7%
- MiniMax M3.1 Flash48%
- GPT-6 Sol47.9%
- Gemini 3.8 Flash47.8%
- Gemini 3.1 Pro Preview47%
- Kimi K346.9%
- Step 5 Preview46.5%
- GPT-6.1 Terra46.2%
- GPT-5.545.8%
- Claude Haiku 5.544.4%
- Muse Spark Open43.9%
- GPT-5.443.7%
- Ling 3.1 Flash43.5%
- DeepSeek V4.1 Flash43.3%
- Grok 4.743.1%
- Qwen3.8 Max43.1%
- GPT-5.6 Terra42.9%
- Grok 4.642.9%
- Grok 4.542.7%
- GPT-5.3-Codex42.5%
- Claude Opus 4.742.3%
- GLM-5.342.3%
- Claude Sonnet 541.3%
- GLM-5.241.1%
- DeepSeek V4 Pro41%
- Gemini 3.6 Flash40.8%
- Claude Opus 4.639.9%
- GLM-5.3-Flash39.9%
- GPT-5.6 Luna39.5%
- GPT-6 Luna38.5%
- MiMo-V2.6-Flash35.1%
- Kimi K2.7 Code35%
- Qwen3.8 27B33.9%
- Claude Sonnet 4.633.6%
- Claude Opus 4.530.1%
- Gemma 4 31B23.6%
- gpt-oss-120b19.6%
- Claude Sonnet 4.517.8%
- Mistral Medium 3.513.8%
- Claude Haiku 4.510.4%
- Llama 4 Maverick4.9%
- Llama 4 Scout3.8%
The other models in the grid have no result on Humanity's Last Exam yet.
Tool useAutomationBench-AA
- Gemini 4 Argon77.5%
- Kimi K3.1 Pro73.3%
- MiniMax M3.1 Flash72.5%
- Claude Sonnet 5.571.8%
- Claude Opus 5.569.5%
- GPT-6 Astra68.5%
- Muse Wonder68.1%
- Ling 3.1 Flash67.5%
- Grok 4.666.7%
- DeepSeek V4.1 Flash66.6%
- Grok 4.765.6%
- GPT-6.1 Sol64.9%
- MiMo-V2.6-Flash64.1%
- GLM-5.362.2%
- GPT-6 Sol61.6%
- GLM-5.3-Flash60.4%
- GPT-6.1 Terra60.2%
- GPT-5.6 Sol60.1%
- Gemini 3.8 Flash59.9%
- GPT-5.6 Terra59.6%
- Claude Fable 5.159.4%
- MiMo-V2.6-Pro58.6%
- Kimi K358.3%
- Grok 4.557.9%
- Muse Spark 1.357.9%
- Mistral Large 457.8%
- DeepSeek V4 Pro56.7%
- Claude Opus 556.6%
- Qwen3.8 Max56.2%
- Claude Fable 554.1%
- GPT-6 Luna53.2%
- Gemini 3.6 Flash53%
- Step 5 Preview51%
- GPT-5.6 Luna50.2%
- Muse Spark Open49.7%
- Qwen3.8 27B48.2%
- GPT-5.547.3%
- Claude Opus 4.845.6%
- Claude Sonnet 536.5%
- Claude Haiku 5.535.4%
- Gemini 3.1 Pro Preview35.4%
- GLM-5.228.4%
- Kimi K2.7 Code24.5%
- Claude Sonnet 4.620.1%
- Claude Sonnet 4.514.1%
- Mistral Medium 3.56.3%
- Gemma 4 31B4.9%
- Claude Haiku 4.53.2%
- gpt-oss-120b0.2%
- Llama 4 Scout0.2%
- Llama 4 Maverick0.2%
The other models in the grid have no result on AutomationBench-AA yet.
Computer useOSWorld 2.1
- Claude Opus 577.7%
- Muse Spark 1.369.6%
- Qwen3.8 27B84.3%Alibaba's own figure, OSWorld-Verified
- Gemini 3.6 Flash83%Google's own figure, OSWorld-Verified
- MiMo-V2.6-Pro82%Xiaomi's own figure, OSWorld-Verified
- Claude Opus 5.581.8%Anthropic's own figure
- MiMo-V2.6-Flash80.8%Xiaomi's own figure, OSWorld-Verified
- Claude Fable 5.180.7%Anthropic's own figure
- Muse Spark Open80.4%Meta's own figure, OSWorld-Verified
- Claude Sonnet 5.580.1%Anthropic's own figure
- Kimi K3.1 Pro79.6%Moonshot AI's own figure, OSWorld 2.1
- GPT-6.1 Sol78%OpenAI's own figure, OSWorld 2.1
- GPT-6 Astra75.8%Meta's run
- GPT-6.1 Terra74.2%OpenAI's own figure, OSWorld 2.1
- Muse Wonder72.6%Meta's own figure
- Claude Haiku 5.572.4%Anthropic's own figure, OSWorld 2.1
- DeepSeek V4.1 Flash69.3%Moonshot AI's run, OSWorld 2.1
- Gemini 4 Argon69.2%Google's own figure, OSWorld 2.0
- GPT-6 Sol64.4%OpenAI's own figure, OSWorld 2.0
- GPT-5.6 Sol62.7%OSWorld team's run, OSWorld 2.0
- MiniMax M3.1 Flash61.7%MiniMax's own figure, OSWorld 2.0
- GLM-5.3-Flash59.1%Z.ai's own figure, OSWorld 2.0
- Gemini 3.8 Flash59%Google's own figure, OSWorld 2.0
- Claude Sonnet 557%Anthropic's own figure
- Claude Opus 4.854.8%OSWorld team's run, OSWorld 2.0
- GPT-5.549.5%OSWorld team's run, OSWorld 2.0
- Claude Opus 4.748.9%OSWorld team's run, OSWorld 2.0
- GPT-6 Luna48.9%Anthropic's run, OSWorld 2.1
- Claude Sonnet 4.641.5%OSWorld team's run, OSWorld 2.0
The other models in the grid have no result on OSWorld 2.1 yet.
Long contextAA-LCR v1.1
- Kimi K388.7%
- Step 5 Preview88.3%
- Kimi K3.1 Pro86.3%
- MiMo-V2.6-Pro86.3%
- Claude Fable 5.185.3%
- Claude Opus 5.584.7%
- GPT-5.584.3%
- GPT-5.6 Sol84%
- GPT-5.6 Luna83.7%
- GPT-6 Sol83.7%
- GPT-6 Luna83.3%
- GPT-5.3-Codex83.3%
- GPT-6.1 Sol83%
- GPT-5.6 Terra83%
- Muse Spark 1.383%
- Claude Haiku 5.582.7%
- Claude Sonnet 5.582.7%
- GPT-6.1 Terra82.7%
- DeepSeek V4.1 Flash82.7%
- Claude Fable 582.3%
- Ling 3.1 Flash82.3%
- Claude Sonnet 582%
- GPT-5.482%
- Gemini 3.1 Pro Preview82%
- Qwen3.8 27B82%
- Gemini 3.8 Flash81.3%
- GPT-6 Astra80.7%
- Muse Wonder80.7%
- DeepSeek V4 Pro80.3%
- Grok 4.680.3%
- Qwen3.8 Max80.3%
- Claude Sonnet 4.680%
- Gemini 3.6 Flash80%
- GLM-5.3-Flash80%
- Gemini 4 Argon79.7%
- MiniMax M3.1 Flash79.7%
- GLM-5.379.7%
- Claude Opus 579.3%
- Kimi K2.7 Code79.3%
- Grok 4.579.3%
- Claude Opus 4.778.7%
- GLM-5.278.3%
- Muse Spark Open78.3%
- Claude Opus 4.678%
- Claude Opus 4.877.7%
- Claude Opus 4.577.3%
- Grok 4.776.7%
- Claude Haiku 4.574.3%
- MiMo-V2.6-Flash74.3%
- Claude Sonnet 4.572.3%
- Gemma 4 31B69.7%
- Mistral Medium 3.569.3%
- gpt-oss-120b52%
- Llama 4 Maverick50%
- Llama 4 Scout27.7%
The other models in the grid have no result on AA-LCR v1.1 yet.
MultimodalMMMU-Pro
- Claude Opus 5.587.7%
- GPT-6 Astra86.9%
- GPT-6.1 Sol86%
- Gemini 3.8 Flash85.6%
- Claude Opus 584.7%
- GPT-6.1 Terra84.7%
- Kimi K3.1 Pro83.9%
- GPT-5.6 Sol83.4%
- Gemini 3.6 Flash83.2%
- GPT-6 Sol82.9%
- Qwen3.8 Max82.8%
- Gemini 3.1 Pro Preview82.4%
- Muse Spark 1.382%
- Muse Spark Open80.9%
- GPT-5.6 Terra80.7%
- DeepSeek V4.1 Flash80.6%
- Kimi K380.5%
- Grok 4.580.4%
- GPT-5.579.9%
- GPT-6 Luna79.7%
- MiniMax M3.1 Flash79.1%
- Claude Opus 4.778.8%
- GPT-5.6 Luna78.6%
- GPT-5.3-Codex78.5%
- GPT-5.478.4%
- Claude Sonnet 577.3%
- Step 5 Preview76.4%
- Qwen3.8 27B76.3%
- Claude Opus 4.675.4%
- Claude Opus 4.574%
- Gemma 4 31B73.4%
- Claude Sonnet 4.673.3%
- MiMo-V2.6-Flash73.1%
- Claude Sonnet 4.568.7%
- Mistral Medium 3.564.9%
- Llama 4 Maverick62.1%
- Claude Haiku 4.558.6%
- Llama 4 Scout52.9%
- Claude Fable 5.186.3%Moonshot AI's run
- GLM-5.3-Flash86%Vals.ai's run, MMMU-Pro
The other models in the grid have no result on MMMU-Pro yet.
FactualityAA-Omniscience Accuracy
- Claude Fable 5.168.4%
- Claude Opus 5.565.5%
- Claude Fable 565.4%
- GPT-6.1 Sol61.8%
- GPT-6 Astra61.6%
- Claude Opus 560.9%
- Gemini 3.1 Pro Preview54.9%
- GPT-6 Sol54.5%
- Claude Sonnet 5.551.5%
- Gemini 3.8 Flash50.5%
- Claude Opus 4.748.9%
- Kimi K3.1 Pro48.9%
- Claude Opus 4.848.8%
- Gemini 4 Argon48.3%
- GPT-6.1 Terra47.6%
- Claude Opus 4.647%
- Claude Opus 4.546.6%
- DeepSeek V4 Pro45.2%
- MiniMax M3.1 Flash45.1%
- Grok 4.742.7%
- GPT-5.6 Terra42.3%
- Muse Spark Open42%
- Claude Sonnet 4.640.9%
- Claude Sonnet 540.1%
- GPT-6 Luna39.6%
- DeepSeek V4.1 Flash39.6%
- Muse Wonder38%
- Muse Spark 1.338%
- Step 5 Preview36.4%
- MiMo-V2.6-Pro34.8%
- Claude Haiku 5.534.7%
- Claude Sonnet 4.532.9%
- GLM-5.332.3%
- Qwen3.8 Max29.5%
- GLM-5.3-Flash28.2%
- MiMo-V2.6-Flash27.2%
- Ling 3.1 Flash26.5%
- Mistral Large 426%
- Mistral Medium 3.523%
- Claude Haiku 4.518%
- Qwen3.8 27B17.3%
- GPT-5.6 Sol69.7%Epoch AI's run, SimpleQA Verified
- Gemini 3.6 Flash66.2%Epoch AI's run, SimpleQA Verified
- GPT-5.563%Epoch AI's run, SimpleQA Verified
- Kimi K350.6%Epoch AI's run, SimpleQA Verified
- Grok 4.648.9%Epoch AI's run, SimpleQA Verified
- Grok 4.548.3%Epoch AI's run, SimpleQA Verified
- GPT-5.6 Luna41%Epoch AI's run, SimpleQA Verified
- Kimi K2.7 Code36.5%Epoch AI's run, SimpleQA Verified
- GLM-5.234.2%Epoch AI's run, SimpleQA Verified
- gpt-oss-120b16.8%OpenAI's own figure, SimpleQA
- Gemma 4 31B10.4%Epoch AI's run, SimpleQA Verified
The other models in the grid have no result on AA-Omniscience Accuracy yet.
Answers in fullSpeechMap
- Grok 4.593.1%
- Gemini 3.6 Flash87.2%
- Gemini 4 Argon87%
- Gemini 3.8 Flash86.1%
- Grok 4.683.6%
- Claude Opus 5.582.8%
- Gemma 4 31B82.3%
- Claude Sonnet 5.581.6%
- Claude Haiku 5.578.8%
- Claude Opus 573.7%
- Claude Fable 5.172.4%
- DeepSeek V4 Pro72%
- GLM-5.371.5%
- Mistral Medium 3.570.5%
- Claude Opus 4.770.2%
- Llama 4 Scout70.1%
- GPT-6.1 Sol69.5%
- GPT-6 Astra68.2%
- Claude Opus 4.867.4%
- MiMo-V2.6-Flash66.9%
- MiMo-V2.6-Pro66.5%
- Kimi K365.9%
- GLM-5.3-Flash65.4%
- Qwen3.8 Max65.3%
- Claude Fable 564.8%
- Grok 4.764.1%
- GPT-6 Sol64%
- GPT-6 Luna63.4%
- Claude Opus 4.659.9%
- Gemini 3.1 Pro Preview59.9%
- Step 5 Preview59.7%
- Llama 4 Maverick58.3%
- Claude Opus 4.557.5%
- Muse Wonder42.7%
- gpt-oss-120b36.3%
- Claude Sonnet 4.635.8%
- Claude Sonnet 4.531.4%
- Muse Spark 1.325.8%
- Claude Haiku 4.524.7%
The other models in the grid have no result on SpeechMap yet.
Instruction followingMultiChallenge
- GPT-6.1 Sol82%
- Gemini 4 Argon78.7%
- DeepSeek V4.1 Flash74.9%
- GPT-6.1 Terra74.2%
- Muse Spark Open74.2%
- Gemini 3.1 Pro Preview71.4%
- Claude Haiku 5.567.7%
- Claude Opus 4.559%
- Claude Sonnet 4.555.3%
- Claude Haiku 4.550.5%
- gpt-oss-120b45.3%
- Claude Opus 4.637.2%
- Qwen3.8 27B79.5%Alibaba's own figure, IFBench
- GPT-5.575.9%Artificial Analysis's run, IFBench
- Gemma 4 31B75.6%Artificial Analysis's run, IFBench
- GPT-5.3-Codex75.4%Artificial Analysis's run, IFBench
- GPT-5.473.9%Artificial Analysis's run, IFBench
- GLM-5.273.3%Artificial Analysis's run, IFBench
- GPT-5.6 Sol72.7%Artificial Analysis's run, IFBench
- GPT-5.6 Terra71.2%Artificial Analysis's run, IFBench
- Mistral Medium 3.568.8%Artificial Analysis's run, IFBench
- Claude Fable 563.5%Artificial Analysis's run, IFBench
- Kimi K2.7 Code63.1%Artificial Analysis's run, IFBench
- Claude Opus 4.862.2%Artificial Analysis's run, IFBench
- Claude Opus 4.758.6%Artificial Analysis's run, IFBench
- Claude Sonnet 4.656.6%Artificial Analysis's run, IFBench
- Llama 4 Maverick43%Artificial Analysis's run, IFBench
- Llama 4 Scout39.5%Artificial Analysis's run, IFBench
The other models in the grid have no result on MultiChallenge yet.
Every measure on record
Cost, speed, reliability, knowledge-work and domain results for Mistral Large 4 beyond the twelve chart axes, each with the run it comes from and its publisher. These stay separate measures; none is folded into the chart.
| Measure | Mistral Large 4 | Run | Source | Read |
|---|---|---|---|---|
| Specs and access | ||||
| Context windowThe most tokens one request can hold, as the maker states it. Size alone says nothing about how well the model uses a long input; the Long context axis measures that. | 1M | Mistral AI | Oct 28, 2026 | |
| Max outputThe most tokens the model writes in one response. | 262K | not published by Mistral; OpenRouter lists 262,144 | OpenRouter | Oct 28, 2026 |
| ReadsThe kinds of input the model accepts. | Text, Images | Artificial Analysis | Oct 27, 2026 | |
| LicenceThe terms the weights or the service are offered under. | Mistral Ecosystem Open Weight License | Mistral AI | Oct 30, 2026 | |
| Open weightsWhether the maker publishes the weights for anyone to run. | Yes | Mistral AI | Oct 30, 2026 | |
| Routes offering itOf the 19 clouds, gateways and coding products GILD tracks, how many offer the release today, preview included. | 3 of 19 | GILD route matrix | Oct 28, 2026 | |
| Cost | ||||
| Answer tokens per taskLower is better. Answer tokens per Intelligence Index task, reasoning excluded. | 17,377 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per task, cache readsLower is better. The cache-read share of AA's cost per Intelligence Index task. | $0.52 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per task, cache writesLower is better. The cache-write share of AA's cost per Intelligence Index task. | $0.77 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per task, inputLower is better. The input-token share of AA's cost per Intelligence Index task. | $1.32 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per task, outputLower is better. The answer-token share of AA's cost per Intelligence Index task. | $0.29 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per task, reasoningLower is better. The reasoning-token share of AA's cost per Intelligence Index task. | $0.22 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost per taskLower is better. What one Intelligence Index task cost AA at list price, input, cache, output and reasoning tokens together. | $1.61 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cost to run the indexLower is better. What the whole Intelligence Index run cost AA at list price. The runs cost from tens of dollars to about $13,000, so the axis is log from $10 to $100,000. | $2,003 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Output tokens per taskLower is better. Tokens the model wrote per Intelligence Index task, reasoning included: how verbose it is. | 69,496 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Reasoning tokens per taskLower is better. Reasoning tokens per Intelligence Index task. | 52,118 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Cached inputLower is better. The maker's list price for input read from its prompt cache, per million tokens. | $0.07 | Mistral AI | Oct 30, 2026 | |
| Speed | ||||
| End-to-end time, 100K promptLower is better. Median seconds for a whole response with a 100,000-token prompt. | 31 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| End-to-end response timeLower is better. Seconds AA measures for a whole response on its standard prompt. | 23 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Time to first answer token, 100K promptLower is better. Median seconds to the first answer token with a 100,000-token prompt. | 27 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Time to first answer tokenLower is better. Seconds until the first answer token, after any reasoning, on AA's API runs. | 19 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Output speed, 100K promptMedian output tokens per second when the prompt is about 100,000 tokens long. | 120 tokens/s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Output speedMedian output tokens per second on AA's API runs, with the 5th and 95th percentile when AA prints them. | 116 tokens/s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Time per taskLower is better. Wall-clock seconds per Intelligence Index task. Runs take from about 15 s to about 33 min, so the axis is log from 1 s to 10,000 s. | 499 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Time to first token, 100K promptLower is better. Median seconds to the first chunk when the prompt is about 100,000 tokens long: long-input latency, separate from window size. | 10 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Time to first tokenLower is better. Median seconds to the first streamed chunk on AA's API runs, with the 5th and 95th percentile when AA prints them. Reasoning models spend this time thinking. | 1.6 s | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Reliability | ||||
| Hallucination rateLower is better. AA-Omniscience hallucination rate shows how often the model answers wrongly when it does not know and should decline. | 28.2% | High effort | Artificial Analysis | Oct 30, 2026 |
| Omniscience accuracyShare of AA-Omniscience knowledge questions answered correctly. | 26% | High effort | Artificial Analysis | Oct 30, 2026 |
| Omniscience indexAA-Omniscience index: correct answers minus wrong answers, declines neutral, -100 to 100. | -5.3 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| SimpleQA Verified (Epoch)Share of SimpleQA Verified factual questions answered correctly, best scorer, Epoch AI's run. A second factuality source beside AA-Omniscience. | 20% | High effort | Epoch AI | Oct 9, 2026 |
| Knowledge work | ||||
| AA-BriefcaseAA-Briefcase overall Elo for long business deliverables, with AA's 95% interval. | 1,391 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| GDP-PDF (AA run)Share of GDP-PDF document tasks where every check passes. | 18.6% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| GDPval-AAGDPval-AA Elo: AA's run of economically valuable knowledge-work tasks, graded pairwise, with AA's interval. | 1,434 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| MLCR (AA run)AA's MLCR overall score for long, multi-document reading. | 7.8% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Capability beyond the chart | ||||
| AA-LCRAA long-context reasoning accuracy. | 81.3% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| ALE-BenchALE-Bench performance rating on long optimization contests, as Epoch AI republishes it. | 1,213 | Sakana AI (via Epoch AI) | Oct 26, 2026 | |
| Chess puzzles (Epoch)Share of chess puzzles solved, Epoch AI's run. | 33% | High effort | Epoch AI | Oct 15, 2026 |
| CritPt (AA run)Share of CritPt research-level physics problems solved in AA's run. | 10.6% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| DTBenchDTBench accuracy, as Epoch AI republishes it. | 88.2% | Redwood Research (via Epoch AI) | Oct 26, 2026 | |
| Epoch Capabilities IndexEpoch Capabilities Index: Epoch AI's composite across its benchmark hub, with its confidence interval. | 150 | Mistral Large 4 | Epoch AI | Oct 26, 2026 |
| FrontierCode (AA run)AA's FrontierCode score: hard, unsaturated coding tasks. | 41.5% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| FrontierMath Tier 4 (Epoch)Share of FrontierMath Tier 4 research math problems solved, Epoch AI's run. | 26.8% | High effort | Epoch AI | Oct 12, 2026 |
| FrontierMath Tiers 1 to 3 (Epoch)Share of FrontierMath Tiers 1 to 3 problems solved, Epoch AI's run. | 58.6% | High effort | Epoch AI | Oct 12, 2026 |
| GPQA Diamond (Epoch)GPQA Diamond accuracy, best scorer, Epoch AI's run. | 88.1% | High effort | Epoch AI | Oct 12, 2026 |
| Humanity's Last Exam (AA run)Share of Humanity's Last Exam questions answered correctly in AA's run. | 35.0% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Intelligence IndexArtificial Analysis Intelligence Index: AA's composite of its own evaluation runs, 0 to 100 points. | 38.4 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| LMCALMCA score, 0 to 100, as Epoch AI republishes it. Its interval is printed only as a half-width, so it is not drawn. | 35.8 | Redwood Research (via Epoch AI) | Oct 26, 2026 | |
| MMMU-Pro (AA run)Multimodal understanding accuracy on MMMU-Pro in AA's run. | 76.4% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Mystery game puzzles (Epoch)Share of mystery game puzzles solved, Epoch AI's run. | 17% | High effort | Epoch AI | Oct 13, 2026 |
| OTIS Mock AIME (Epoch)Share of OTIS Mock AIME 2024 to 2025 problems solved, Epoch AI's run. | 98.3% | High effort | Epoch AI | Oct 12, 2026 |
| ProofBenchShare of ProofBench proofs graded correct, from Vals AI as Epoch AI republishes it. | 10% | Vals AI (via Epoch AI) | Oct 26, 2026 | |
| SciCode (AA run)SciCode subproblem accuracy in AA's run. | 54.2% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Terminal-Bench 4.0 (AA run)Share of Terminal-Bench 4.0 tasks solved in AA's own harness. | 26.8% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Terminal-Bench Science (AA run)Share of Terminal-Bench Science tasks solved in AA's harness. | 4.8% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| WeirdML v3, best runWeirdML v3 maximum performance across runs, as Epoch AI republishes it. | 12.2% | High effort | Håvard Tveit Ihle (via Epoch AI) | Oct 26, 2026 |
| Domains | ||||
| EconomicsArtificial Analysis capability score for economics questions, 0 to 100 points. | 43.6 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| EngineeringArtificial Analysis capability score for engineering questions, 0 to 100 points. | 37.6 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Finance and accountingArtificial Analysis capability score for finance and accounting questions, 0 to 100 points. | 38.3 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Healthcare and medicalArtificial Analysis capability score for healthcare and medical questions, 0 to 100 points. | 34.9 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| LegalArtificial Analysis capability score for legal questions, 0 to 100 points. | 37 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Strategy and operationsArtificial Analysis capability score for strategy and operations questions, 0 to 100 points. | 42.2 | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
| Harvey Legal Lab (AA run)AA's run of Harvey's legal task lab. | 3.1% | Mistral Large 4 Preview | Artificial Analysis | Oct 27, 2026 |
Routes, prices and uptime
Where Mistral Large 4 is offered, what each route charges, and how each endpoint is holding up.
Where it is offered
3 of 19 routes GILD tracks offer Mistral Large 4 today, preview included. Read Oct 28, 2026 from each route's own pages.
| Route | Status | Model ID | Limits and lifecycle | Sources |
|---|---|---|---|---|
| The maker's own API | Preview | mistral-large-4 | Mistral's model page, read Oct 30, 2026, dates Mistral Large 4 Oct 6, 2026 and labels it Public Preview; 1M context; US$0.70 input, US$0.07 cached input and US$2.10 output per million tokens; batching through /v1/batch. On Oct 28, 2026 the same page listed US$1.36 input and US$4.18 output. | Mistral AI, Mistral Large 4Mistral AI, Mistral Large 4 (Le Chonk) is now open weight |
| GitHub Models | Retired | not applicableservice retired | service retired July 30, 2026 | GitHub, GitHub Models |
| OpenCode | Available | mistral-large-4 | OpenCode Zen chat-completions endpoint | Anomaly (OpenCode), ModelsOpenCode (Anomaly) |
| OpenRouter | Preview | mistralai/mistral-large-4-0 | OpenRouter's models API, read Oct 28, 2026, lists mistralai/mistral-large-4-0 as Mistral Large 4, with a 1,048,576-token context and 262,144 max output tokens. Mistral's model page, read Oct 30, 2026, labels Mistral Large 4 Public Preview, and Mistral's open-weight announcement, read Oct 30, 2026, names OpenRouter as a place to start. | OpenRouterMistral AI, Mistral Large 4Mistral AI, Mistral Large 4 (Le Chonk) is now open weight |
Not offered on 6 routes: Amazon Bedrock, Antigravity, Cursor, GitHub Copilot, Google Cloud Vertex / Agent Platform (Google managed) and Microsoft Foundry.
9 routes do not say: Claude Code, Claude on Google Cloud (Anthropic managed), Claude Platform on AWS, Codex, Databricks, Gemini CLI, Grok Build, Snowflake Cortex and Windsurf / Devin.
What each of these 15 routes' pages say, with sources
| Route | Status | Model ID | Limits and lifecycle | Sources |
|---|---|---|---|---|
| Amazon Bedrock | Not offered | Bedrock lists Mistral Large and Mistral Large 3 | Amazon Web Services, Models at a glanceAmazon Web Services, Supported foundation models in Amazon Bedrock | |
| Antigravity | Not offered | Antigravity lists Gemini, Claude and GPT-OSS models only | ||
| Cursor | Not offered | absent from Cursor's model table | Anysphere (Cursor)Cursor | |
| GitHub Copilot | Not offered | Absent from the Copilot supported-models table, which lists each supported model | GitHub, Supported AI models in GitHub Copilot | |
| Google Cloud Vertex / Agent Platform (Google managed) | Not offered | Google lists Mistral Medium 3, OCR, Small 3.1, Codestral 2 | Google Cloud, Anthropic's Claude on Google Cloud modelsGoogle Cloud, Gemini Enterprise Agent Platform partner models for MaaS | |
| Microsoft Foundry | Not offered | Foundry lists Mistral-Large-3 | Microsoft Learn, Foundry Models sold by Azure | |
| Claude Code | Not stated | Claude Code's model configuration page, read Oct 28 and Oct 29, 2026, does not name this model. It takes a full model name on the Anthropic API, an inference profile ARN on Amazon Bedrock and a deployment name on Microsoft Foundry. | Anthropic | |
| Claude on Google Cloud (Anthropic managed) | Not stated | Region, tier and limits not stated on the route's pages. | ||
| Claude Platform on AWS | Not stated | Region, tier and limits not stated on the route's pages. | ||
| Codex | Not stated | Codex docs list OpenAI models only | ||
| Databricks | Not stated | Databricks list names no Mistral model | Databricks | |
| Gemini CLI | Not stated | Region, tier and limits not stated on the route's pages. | ||
| Grok Build | Not stated | Grok Build pages name only grok-4.7 | xAI (SpaceXSI) | |
| Snowflake Cortex | Not stated | Region, tier and limits not stated on the route's pages. | ||
| Windsurf / Devin | Not stated | absent from Devin model data | Cognition (Devin, formerly Windsurf), AI Models |
Prices by route
The lowest standard rate is Mistral API, at $0.7 / $2.10 per million input and output tokens. US dollars per million tokens, as each route lists them, one row per route and rate plan, cheapest standard rate first after the maker's own; a blended rate states its mix. OpenRouter's endpoints follow under one heading, each with its monitor's uptime over 1 day and 30 minutes and the context and output it serves.
| Route | Standard, in / out | Cache read | Other rates | Source, or uptime (1 day / 30 min) and limits |
|---|---|---|---|---|
| Mistral APIMakerLowest | $0.7 / $2.10 | $0.07 | Mistral AI, Oct 30, 2026 | |
| Artificial Analysis price record | $1.36 / $4.18 | $0.14 | Blended $2.06, 3 input : 1 output, no cacheBlended $0.79, 7 cache-hit : 2 input : 1 outputCache write $1.36 | Artificial Analysis, Oct 27, 2026 |
| OpenRouter | $1.36 / $4.18 | $0.14 | OpenRouter, Oct 28, 2026 | |
| Through OpenRouter, 3 provider endpoints. OpenRouter, Oct 28, 2026 | ||||
| Mistral | $1.36 / $4.18 | $0.14 | Up 99.9% / 100%1.05M context, 262K output | |
| Mistral (zdr) | $1.36 / $4.18 | $0.14 | Up 99.7% / 100%1.05M context, 262K output | |
| Mistral (eu) | $1.50 / $4.60 | $0.15 | Up 100%524K context, 262K output | |
Uptime and speed by endpoint
Each provider endpoint as its monitor reports it: uptime over the last day and the last 30 minutes, output speed and time to first token, and the limits it serves. A blank is a figure the monitor does not publish for that endpoint.
| Endpoint | Uptime, 1 day | Uptime, 30 min | Output speed | First token | Context / max output | Source |
|---|---|---|---|---|---|---|
| Artificial Analysis: Mistral | 114 tokens/s | 1.6 s | Artificial Analysis, Oct 28, 2026 |
Lineup
Mistral AI's 2 releases in 2 families, oldest to newest along each tier, with list price and speed. Mistral Large 4 is lit.
CurrentLegacyThis model
Price is the list price per 1M tokens, input / output. Speed is median output tokens per second. Bars share one scale across the lineup. Read between Oct 28, 2026 and Oct 30, 2026.
Mistral Large
Mistral AI's flagship general-purpose model family.
1 release
- This page4Current. In 1 product we coverPrice$0.70 / $2.10Speed161.3 tok/s
Mistral Medium
Mistral AI's mid-size model family for agentic and coding work.
1 release
Where it fits
The layers of Agentic Software Development that Mistral Large 4 can power. A product supplies the rest.
Product & Engineering, Software Development
Agentic Software Development
- Primary role
- Foundation Model / Reasoning Engine
Mistral Large 4 can power coding, reasoning, and validation work inside an agentic software-development product. The Model does not itself provide the surrounding interface, execution environment, or system of record.
- Supporting
Work planning & task decomposition
The Model can assist with work planning & task decomposition when a Product supplies the necessary context and tools.
- Strong fit
Codebase context & intelligence
The provider documents long-context or codebase-oriented reasoning suitable for this layer. Mistral AI's page
- Strong fit
Coding & implementation
The provider documents this Model for software-engineering and coding work. Mistral AI's page
- Supporting
Debugging & investigation
The Model can assist with debugging & investigation when a Product supplies the necessary context and tools.
- Supporting
Testing & verification
The Model can assist with testing & verification when a Product supplies the necessary context and tools.
- Supporting
Code review & quality
The Model can assist with code review & quality when a Product supplies the necessary context and tools.
Products
We cover 1 product that offers Mistral Large 4 today, all subscriptions.
Comparisons
GILD comparisons that name Mistral Large 4, then those of the clouds and APIs that serve it.
- DeepSeek API vs OpenRouter: Buy DeepSeek From DeepSeek, or From Another Host?Compares routes that serve Mistral Large 4
- LiteLLM vs OpenRouter: Run Your Own Gateway or Rent One?Compares routes that serve Mistral Large 4
- OpenRouter vs Portkey: Rent Model Access or Run a Governed Gateway?Compares routes that serve Mistral Large 4
- Together AI vs OpenRouter: One Open-Model Host or a Router Across HostsCompares routes that serve Mistral Large 4
What practitioners report
Strongest on Quality and reasoning, weakest on Developer experience.
Each figure is the share of accounts naming Mistral Large 4 that speak well of it on that axis. An axis is scored once praise and concerns are both on record from at least five accounts by three authors. An account from the maker, a company that sells or serves the model, or the maker's expert program is listed and counts toward no figure.
| Axis | Favorable | Accounts |
|---|---|---|
| Quality and reasoningWhether its answers and code are right on the author's own task: reasoning, accuracy, instruction following, and consistency. | 58% | 11 favorable, 8 not, 18 authors |
| Tool use and structured outputWhether it calls tools correctly, returns valid structured output, and keeps agent loops on track. | 38% | 5 favorable, 8 not, 13 authors |
| Long contextHow it holds up over long documents, large codebases, long sessions, and many images or PDFs. | Needs 5 accounts from 3 authors | 1 favorable, 3 not, 4 authors |
| Developer experienceHow it is to build with: prompting, steering, effort settings, and support across SDKs and providers. | 13% | 2 favorable, 14 not, 16 authors |
| CostWhat the work costs: price per task, token use, and what caching and batch save. | 27% | 3 favorable, 8 not, 11 authors |
| SpeedResponse time, time to first token, and throughput. | 21% | 3 favorable, 11 not, 14 authors |
| ReliabilityWhether requests keep succeeding: overloaded errors, outages, rate limits, and changes that broke what worked. | 38% | 3 favorable, 5 not, 8 authors |
Every model on these seven axes
| Axis | Mistral Large 4 | Mistral Medium 3.5 | Claude Haiku 5.5 | Claude Sonnet 5.5 | Claude Opus 5.5 | Claude Fable 5.1 | Claude Opus 5 | Claude Sonnet 5 | Claude Fable 5 | Claude Opus 4.8 | Claude Opus 4.7 | Claude Sonnet 4.6 | Claude Opus 4.6 | Claude Opus 4.5 | Claude Haiku 4.5 | Claude Sonnet 4.5 | GPT-6.1 Terra | GPT-6.1 Sol | GPT-6 Luna | GPT-6 Astra | GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | GPT-5.5 | GPT-5.4 | gpt-oss-120b | GPT-6 Sol | GPT-5.3-Codex | Gemini 4 Argon | Gemini 3.8 Flash | Gemma 4 31B | Gemini 3.1 Pro Preview | Gemini 3.6 Flash | DeepSeek V4.1 Flash | DeepSeek V4 Pro | Kimi K3.1 Pro | Kimi K3 | Kimi K2.7 Code | Grok 4.7 | Grok 4.6 | Grok 4.5 | MiniMax M3.1 Flash | Mercury 3 | GLM-5.3-Flash | GLM-5.3 | GLM-5.2 | Muse Spark Open | Muse Wonder | Muse Spark 1.3 | Llama 4 Scout | Llama 4 Maverick | Qwen3.8 27B | Qwen3.8 Max | MiMo-V2.6-Pro | MiMo-V2.6-Flash | Ling 3.1 Flash | Step 5 Preview |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Quality and reasoning | 58%11 favorable, 8 not, 18 authors | 36%5 favorable, 9 not, 14 authors | 73%19 favorable, 7 not, 26 authors | 64%14 favorable, 8 not, 22 authors | 68%17 favorable, 8 not, 25 authors | 54%15 favorable, 13 not, 28 authors | 50%6 favorable, 6 not, 12 authors | 56%9 favorable, 7 not, 16 authors | 68%13 favorable, 6 not, 18 authors | 68%17 favorable, 8 not, 25 authors | 57%12 favorable, 9 not, 21 authors | 67%12 favorable, 6 not, 18 authors | 70%19 favorable, 8 not, 27 authors | 62%13 favorable, 8 not, 20 authors | 58%14 favorable, 10 not, 24 authors | 69%11 favorable, 5 not, 16 authors | 33%3 favorable, 6 not, 8 authors | 55%12 favorable, 10 not, 22 authors | 55%16 favorable, 13 not, 29 authors | 44%12 favorable, 15 not, 26 authors | 33%3 favorable, 6 not, 9 authors | 38%3 favorable, 5 not, 8 authors | Needs 5 accounts from 3 authors3 favorable, 1 not, 4 authors | 33%4 favorable, 8 not, 12 authors | 33%2 favorable, 4 not, 6 authors | 58%22 favorable, 16 not, 38 authors | 31%10 favorable, 22 not, 32 authors | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | 63%22 favorable, 13 not, 35 authors | 56%27 favorable, 21 not, 45 authors | 80%24 favorable, 6 not, 28 authors | 56%20 favorable, 16 not, 36 authors | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | 61%25 favorable, 16 not, 40 authors | Only concerns on record0 favorable, 4 not, 4 authors | 59%20 favorable, 14 not, 32 authors | Only praise on record3 favorable, 0 not, 3 authors | 50%3 favorable, 3 not, 6 authors | 43%16 favorable, 21 not, 36 authors | 86%6 favorable, 1 not, 7 authors | 60%3 favorable, 2 not, 5 authors | 62%13 favorable, 8 not, 21 authors | 86%18 favorable, 3 not, 21 authors | 67%14 favorable, 7 not, 21 authors | 76%47 favorable, 15 not, 61 authors | 71%5 favorable, 2 not, 7 authors | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | 20%1 favorable, 4 not, 5 authors | Only praise on record2 favorable, 0 not, 2 authors | 53%8 favorable, 7 not, 15 authors | 44%8 favorable, 10 not, 17 authors | 71%24 favorable, 10 not, 34 authors | 57%4 favorable, 3 not, 7 authors | 22%2 favorable, 7 not, 9 authors | 40%4 favorable, 6 not, 10 authors | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors |
| Tool use and structured output | 38%5 favorable, 8 not, 13 authors | Only concerns on record0 favorable, 5 not, 5 authors | 67%6 favorable, 3 not, 9 authors | 40%4 favorable, 6 not, 10 authors | 46%6 favorable, 7 not, 13 authors | 30%3 favorable, 7 not, 10 authors | Needs 5 accounts from 3 authors3 favorable, 1 not, 4 authors | 70%7 favorable, 3 not, 10 authors | 57%4 favorable, 3 not, 7 authors | 50%4 favorable, 4 not, 8 authors | 50%4 favorable, 4 not, 8 authors | 29%2 favorable, 5 not, 7 authors | 33%4 favorable, 8 not, 12 authors | 70%7 favorable, 3 not, 10 authors | 43%3 favorable, 4 not, 6 authors | 50%3 favorable, 3 not, 6 authors | 40%2 favorable, 3 not, 5 authors | 40%4 favorable, 6 not, 10 authors | 33%3 favorable, 6 not, 9 authors | 62%8 favorable, 5 not, 13 authors | No accounts yet | Only concerns on record0 favorable, 2 not, 2 authors | Only praise on record1 favorable, 0 not, 1 author | Only praise on record1 favorable, 0 not, 1 author | No accounts yet | 27%4 favorable, 11 not, 15 authors | 56%5 favorable, 4 not, 9 authors | No accounts yet | 25%2 favorable, 6 not, 8 authors | 47%7 favorable, 8 not, 15 authors | 67%4 favorable, 2 not, 6 authors | 54%7 favorable, 6 not, 12 authors | Only concerns on record0 favorable, 1 not, 1 author | 59%10 favorable, 7 not, 17 authors | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors3 favorable, 1 not, 4 authors | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | 38%6 favorable, 10 not, 16 authors | Only praise on record2 favorable, 0 not, 2 authors | Only concerns on record0 favorable, 1 not, 1 author | 67%4 favorable, 2 not, 6 authors | 80%8 favorable, 2 not, 10 authors | 33%2 favorable, 4 not, 6 authors | 57%4 favorable, 3 not, 7 authors | No accounts yet | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | Needs 5 accounts from 3 authors1 favorable, 2 not, 3 authors | Only concerns on record0 favorable, 4 not, 4 authors | 46%6 favorable, 7 not, 13 authors | Only praise on record1 favorable, 0 not, 1 author | Only concerns on record0 favorable, 5 not, 5 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author |
| Long context | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | No accounts yet | 29%2 favorable, 5 not, 7 authors | 60%3 favorable, 2 not, 5 authors | 80%8 favorable, 2 not, 10 authors | 63%5 favorable, 3 not, 8 authors | 67%4 favorable, 2 not, 6 authors | 50%3 favorable, 3 not, 6 authors | 33%2 favorable, 4 not, 6 authors | 60%6 favorable, 4 not, 10 authors | 50%4 favorable, 4 not, 7 authors | 71%5 favorable, 2 not, 7 authors | 64%7 favorable, 4 not, 11 authors | 40%4 favorable, 6 not, 10 authors | 40%2 favorable, 3 not, 5 authors | 50%4 favorable, 4 not, 8 authors | Only concerns on record0 favorable, 1 not, 1 author | 38%3 favorable, 5 not, 8 authors | 20%1 favorable, 4 not, 5 authors | 27%3 favorable, 8 not, 11 authors | No accounts yet | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | 25%2 favorable, 6 not, 8 authors | Only concerns on record0 favorable, 3 not, 3 authors | No accounts yet | 25%2 favorable, 6 not, 8 authors | 38%5 favorable, 8 not, 13 authors | 20%1 favorable, 4 not, 5 authors | 44%7 favorable, 9 not, 16 authors | Only praise on record1 favorable, 0 not, 1 author | 58%7 favorable, 5 not, 12 authors | No accounts yet | Needs 5 accounts from 3 authors3 favorable, 1 not, 4 authors | Only praise on record1 favorable, 0 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | 42%5 favorable, 7 not, 12 authors | No accounts yet | No accounts yet | 40%2 favorable, 3 not, 5 authors | 60%3 favorable, 2 not, 5 authors | No accounts yet | 40%2 favorable, 3 not, 5 authors | No accounts yet | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 3 not, 3 authors | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | 75%6 favorable, 2 not, 8 authors | Only praise on record1 favorable, 0 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | No accounts yet | No accounts yet |
| Developer experience | 13%2 favorable, 14 not, 16 authors | Only concerns on record0 favorable, 9 not, 9 authors | 25%2 favorable, 6 not, 8 authors | 38%5 favorable, 8 not, 13 authors | 38%6 favorable, 10 not, 16 authors | 50%7 favorable, 7 not, 14 authors | 25%3 favorable, 9 not, 12 authors | 29%2 favorable, 5 not, 7 authors | 25%3 favorable, 9 not, 12 authors | 53%10 favorable, 9 not, 19 authors | 29%4 favorable, 10 not, 14 authors | 57%4 favorable, 3 not, 7 authors | 45%5 favorable, 6 not, 11 authors | 67%8 favorable, 4 not, 12 authors | Needs 5 accounts from 3 authors2 favorable, 2 not, 4 authors | 50%3 favorable, 3 not, 6 authors | Only concerns on record0 favorable, 2 not, 2 authors | 27%3 favorable, 8 not, 11 authors | 25%2 favorable, 6 not, 8 authors | 41%7 favorable, 10 not, 17 authors | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | No accounts yet | No accounts yet | 14%1 favorable, 6 not, 7 authors | 25%3 favorable, 9 not, 12 authors | No accounts yet | 33%3 favorable, 6 not, 9 authors | 39%7 favorable, 11 not, 17 authors | Only concerns on record0 favorable, 7 not, 7 authors | 40%8 favorable, 12 not, 20 authors | No accounts yet | 35%8 favorable, 15 not, 23 authors | No accounts yet | 9%1 favorable, 10 not, 11 authors | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | 25%5 favorable, 15 not, 20 authors | No accounts yet | No accounts yet | Only concerns on record0 favorable, 7 not, 7 authors | 11%1 favorable, 8 not, 9 authors | Only concerns on record0 favorable, 3 not, 3 authors | 33%5 favorable, 10 not, 15 authors | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | Only concerns on record0 favorable, 6 not, 6 authors | Only concerns on record0 favorable, 1 not, 1 author | 17%1 favorable, 5 not, 6 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | No accounts yet |
| Cost | 27%3 favorable, 8 not, 11 authors | Only concerns on record0 favorable, 8 not, 8 authors | 69%20 favorable, 9 not, 29 authors | 45%10 favorable, 12 not, 22 authors | 56%15 favorable, 12 not, 27 authors | 20%4 favorable, 16 not, 20 authors | 43%3 favorable, 4 not, 7 authors | 39%7 favorable, 11 not, 17 authors | 8%1 favorable, 11 not, 12 authors | 40%6 favorable, 9 not, 14 authors | 50%7 favorable, 7 not, 13 authors | 50%6 favorable, 6 not, 12 authors | 33%7 favorable, 14 not, 20 authors | 53%8 favorable, 7 not, 15 authors | 29%4 favorable, 10 not, 12 authors | 38%3 favorable, 5 not, 8 authors | 80%8 favorable, 2 not, 9 authors | 74%14 favorable, 5 not, 19 authors | 79%22 favorable, 6 not, 26 authors | 29%7 favorable, 17 not, 23 authors | 57%4 favorable, 3 not, 7 authors | Only concerns on record0 favorable, 1 not, 1 author | 50%3 favorable, 3 not, 6 authors | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | 63%5 favorable, 3 not, 8 authors | 60%3 favorable, 2 not, 5 authors | 55%12 favorable, 10 not, 21 authors | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | 35%7 favorable, 13 not, 20 authors | 58%14 favorable, 10 not, 23 authors | Only praise on record7 favorable, 0 not, 7 authors | 50%8 favorable, 8 not, 16 authors | Only concerns on record0 favorable, 1 not, 1 author | 66%25 favorable, 13 not, 37 authors | Only praise on record2 favorable, 0 not, 2 authors | 40%10 favorable, 15 not, 25 authors | 71%5 favorable, 2 not, 7 authors | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | 19%5 favorable, 21 not, 26 authors | 20%1 favorable, 4 not, 5 authors | Only praise on record4 favorable, 0 not, 4 authors | 80%12 favorable, 3 not, 15 authors | 50%6 favorable, 6 not, 12 authors | 76%16 favorable, 5 not, 21 authors | 35%8 favorable, 15 not, 23 authors | 71%5 favorable, 2 not, 7 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 9 not, 9 authors | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | Only praise on record1 favorable, 0 not, 1 author | Only praise on record1 favorable, 0 not, 1 author | Needs 5 accounts from 3 authors2 favorable, 2 not, 4 authors | Only concerns on record0 favorable, 2 not, 2 authors | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | Only praise on record3 favorable, 0 not, 3 authors | Only praise on record1 favorable, 0 not, 1 author | Only praise on record1 favorable, 0 not, 1 author |
| Speed | 21%3 favorable, 11 not, 14 authors | Only concerns on record0 favorable, 2 not, 2 authors | 74%14 favorable, 5 not, 19 authors | 88%14 favorable, 2 not, 16 authors | 46%6 favorable, 7 not, 13 authors | 44%4 favorable, 5 not, 9 authors | 60%3 favorable, 2 not, 5 authors | 44%4 favorable, 5 not, 9 authors | 57%4 favorable, 3 not, 7 authors | 31%4 favorable, 9 not, 13 authors | 63%5 favorable, 3 not, 8 authors | 17%1 favorable, 5 not, 6 authors | 36%4 favorable, 7 not, 11 authors | 73%8 favorable, 3 not, 11 authors | 78%7 favorable, 2 not, 9 authors | 75%6 favorable, 2 not, 8 authors | 83%5 favorable, 1 not, 5 authors | 37%7 favorable, 12 not, 19 authors | 45%5 favorable, 6 not, 11 authors | 41%7 favorable, 10 not, 17 authors | Only praise on record1 favorable, 0 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | 80%4 favorable, 1 not, 5 authors | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | No accounts yet | 85%23 favorable, 4 not, 27 authors | 57%4 favorable, 3 not, 7 authors | Only praise on record1 favorable, 0 not, 1 author | 45%5 favorable, 6 not, 11 authors | 61%19 favorable, 12 not, 31 authors | 53%10 favorable, 9 not, 17 authors | 35%6 favorable, 11 not, 16 authors | Needs 5 accounts from 3 authors2 favorable, 1 not, 3 authors | 68%21 favorable, 10 not, 30 authors | Only concerns on record0 favorable, 1 not, 1 author | 21%4 favorable, 15 not, 19 authors | Only concerns on record0 favorable, 2 not, 2 authors | No accounts yet | 29%5 favorable, 12 not, 17 authors | Needs 5 accounts from 3 authors1 favorable, 2 not, 3 authors | Only praise on record2 favorable, 0 not, 2 authors | 44%4 favorable, 5 not, 9 authors | 65%13 favorable, 7 not, 20 authors | 33%3 favorable, 6 not, 9 authors | 20%3 favorable, 12 not, 15 authors | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | Needs 5 accounts from 3 authors3 favorable, 1 not, 4 authors | No accounts yet | 33%7 favorable, 14 not, 21 authors | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | Only concerns on record0 favorable, 4 not, 4 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 2 not, 2 authors | Only concerns on record0 favorable, 1 not, 1 author |
| Reliability | 38%3 favorable, 5 not, 8 authors | Only concerns on record0 favorable, 4 not, 4 authors | 20%1 favorable, 4 not, 5 authors | 18%2 favorable, 9 not, 11 authors | 13%1 favorable, 7 not, 8 authors | 9%1 favorable, 10 not, 11 authors | 13%1 favorable, 7 not, 8 authors | 63%5 favorable, 3 not, 7 authors | 22%2 favorable, 7 not, 9 authors | 17%2 favorable, 10 not, 12 authors | 18%2 favorable, 9 not, 11 authors | 17%1 favorable, 5 not, 6 authors | 20%3 favorable, 12 not, 15 authors | 38%6 favorable, 10 not, 16 authors | 29%2 favorable, 5 not, 7 authors | 40%4 favorable, 6 not, 10 authors | Only concerns on record0 favorable, 1 not, 1 author | 13%1 favorable, 7 not, 8 authors | Only concerns on record0 favorable, 5 not, 5 authors | 14%2 favorable, 12 not, 14 authors | Only concerns on record0 favorable, 4 not, 4 authors | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | Only concerns on record0 favorable, 2 not, 2 authors | Only concerns on record0 favorable, 4 not, 4 authors | Only concerns on record0 favorable, 4 not, 4 authors | 17%1 favorable, 5 not, 6 authors | Only concerns on record0 favorable, 3 not, 3 authors | 5%1 favorable, 18 not, 19 authors | 15%3 favorable, 17 not, 20 authors | Only concerns on record0 favorable, 5 not, 5 authors | 14%3 favorable, 19 not, 22 authors | Only concerns on record0 favorable, 1 not, 1 author | 42%5 favorable, 7 not, 12 authors | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | 8%1 favorable, 12 not, 13 authors | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 1 not, 2 authors | 22%2 favorable, 7 not, 9 authors | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | 60%3 favorable, 2 not, 5 authors | 33%2 favorable, 4 not, 5 authors | Only concerns on record0 favorable, 7 not, 7 authors | Only concerns on record0 favorable, 4 not, 4 authors | Only concerns on record0 favorable, 1 not, 1 author | No accounts yet | Only concerns on record0 favorable, 2 not, 2 authors | Only concerns on record0 favorable, 1 not, 1 author | Needs 5 accounts from 3 authors1 favorable, 3 not, 4 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 9 not, 9 authors | Only concerns on record0 favorable, 1 not, 1 author | Only concerns on record0 favorable, 2 not, 2 authors | No accounts yet | No accounts yet | Only concerns on record0 favorable, 1 not, 1 author |
The 67 accounts behind Mistral Large 4's figures
GILD searched Hacker News, GitHub, Reddit, X, Artificial Analysis, OpenRouter, LMArena, Vals AI and the web for accounts that name Mistral Large 4.
Quality and reasoning 58%
- Favorable
above average in intelligence
Artificial Analysis - Mistral Large 4. Artificial Analysis, independent benchmark publisher, self-described, Oct 26, 2026. Mistral Large 4 (reasoning).
- Favorable
it gave me quite a lot of good info during a half an hour chat on C and embedded programming
Mistral Large 4. delis-thumbs-7e, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through OpenRouter in a web UI, a half-hour chat on C and embedded programming.
- Unfavorable
it keeps trying to cheat at the task (trying to remember hallucinations)
Mistral Large 4. TuxSH, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 running as subagents.
- Favorable
Definitely the best I've seen from any Mistral model
Mistral Large 4. simonw (Simon Willison), Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through the llm CLI at reasoning none and high, pelican SVG test.
- Favorable
the leap from Mistral Large 3 (their previous best model) to Mistral Large 4 was similar to the Haiku 4.5 to 5.5 jump
Claude Haiku 5.5. simonw (Simon Willison), Hacker News commenter, self-described, Oct 8, 2026. Mistral Large 4 against Mistral Large 3 on the pelican SVG test.
- Unfavorable
verbose and also performse worse
Mistral Large 4. XCSme, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 against GPT-6 Luna on the aibenchy comparison.
- Favorable
I just runned it on code reviews right now and it was able to catch some thread safety issue than DeepSeek-4.1 didn't
Mistral Large 4. adev_, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 on code reviews.
- Unfavorable
Mistral Large 4 is not very good. It can sometimes solve a game with ~40 guesses whereas the top models like Gemini 3.8 Flash or Grok 4.7 can one shot most puzzles
Mistral Large 4. pampas, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 on the Redactle trivia benchmark.
- Favorable
goes from 58% to 74% correct. Definitely a generational shift
Mistral Large 4. chriddyp (Chris Parmer), co-founder of Plotly, Oct 6, 2026. Mistral Large 4 on Plotly's data analytics benchmark.
- Unfavorable
it performed pretty poorly (on par with Kimi K3)
Mistral Large 4. dom96, Hacker News commenter, self-described, Oct 21, 2026. Mistral Large 4 on the killswitch-lang adversarial programming language benchmark.
- Unfavorable
it seemed to make all kinds of obvious errors of the kind that I haven't seen from opus, sol, deepseek flash etc in over a year
Timestral: Mistral's time-series foundation models. ritzaco, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 (Le Chonk) for general coding work.
- Unfavorable
it spits out Chinese characters just like the chinese models when chatting with it in German
Mistral Large 4. nik736, Hacker News commenter, self-described, Oct 21, 2026. Mistral Large 4 through the API, chats held with German prompts.
- Favorable
small mistakes but reached the solution
Samsung TV privacy issues. nekitamo, Hacker News commenter, self-described, Oct 24, 2026. Mistral Large 4 (listed as Mistral Large 4.0) on a crackme reverse engineering benchmark through Ghidrasql.
- Unfavorable
It's not the best, the cheapest, or the fastest.
Pawel Huryn on X: Mistral Large 4 bug-fix test. Paweł Huryn, AI PM and author of the Product Compass newsletter, self-described, Oct 7, 2026. Mistral Large 4 at high reasoning on two real repos with 105 replanted bugs.
- Favorable
it knows more, good job
H. Gskrd on X: testing Mistral Large 4. h_gskrd, X user, self-described, Oct 14, 2026. Mistral Large 4, which the author calls GA and therefore usable with zero data retention; Mistral's model card lists it as a public preview, and Mistral's Commercial Terms (section 4.3) say zero data retention opt-outs do not apply to preview models.
- Favorable
Mistral Large 4 joins my leaderboard right beside those "almost frontier" models!
Povilas Korop on X: Mistral Large 4 on the AI Coding Daily leaderboard. Povilas Korop, AI coding and Laravel educator, self-described, Oct 15, 2026. Mistral Large 4 on the AI Coding Daily coding leaderboard.
- Favorable
This makes it the most intelligent model from outside the US and China
Mistral has released Mistral Large 4, making France home to the most intelligent model outside the US and China. Artificial Analysis, independent benchmark publisher, self-described, Oct 6, 2026. Mistral Large 4 Preview on the Artificial Analysis Intelligence Index, Cyber Index and GDP.pdf.
- Favorable
Mistral Large 4 passes 32 of 43 (74%, a C), and it improved in every category but one
Mistral Large 4 Benchmark Results for Data Analysis. Plotly, company blog by Chris Parmer, co-founder and CPO, Oct 6, 2026. Mistral Large 4 in Plotly Studio Embedded on a 43-question agentic data analytics exam over a 20M-row warehouse.
- Unfavorable
SWE-bench Multimodal 33.13 % ± 2.15 20 / 20
Mistral Large 4 Benchmarks, Cost and Capabilities | Vals AI. Vals AI, independent benchmark publisher, self-described, Oct 26, 2026. Mistral Large 4 at high reasoning effort through the Mistral API on Vals AI benchmarks.
- Favorable
Mistral Large 4 just tied #1 on 669 clinical decisions
Maziyar Panahi on X: Mistral Large 4 on clinical decisions. Maziyar Panahi, on-device clinical AI builder (X bio), self-described, Oct 6, 2026. Mistral Large 4 on his 669-decision clinical benchmark on launch day.
- Unfavorable
On my chess bench, where general capability increase usually leads to higher elos for new models, it performed abysmal and still near mistral-large-2-2411 (nov 2024) levels.
Europe rejoins the fight with Chonky! Mistral Large 4 Released, Open weights end of month, who's ready?. dubesor86, Reddit commenter who runs a chess benchmark, self-described, Oct 7, 2026. Mistral Large 4 on code analysis, image classification and his chess benchmark.
- Favorable
None of the 14 fails is a planning defect
docs: add the mistral-large-4 test results to the mistral backlog. Rabbid76, GitHub user and roomle-hi-example contributor, self-described, Oct 21, 2026. mistral-large-4 on 52 of 75 MCP planning tests against the roomle UI.
Tool use and structured output 38%
- Unfavorable
I've had seen subagents fail with "hmm, hmm - hmm, hmm (...)" multiple times
Mistral Large 4. TuxSH, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 running as subagents.
- Unfavorable
Seems to have problems with tool calling? When doing heavy research it makes a 'json db' but then has trouble working with jq.
Mistral Large 4. postepowanieadm, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 on a heavy research task with tools.
- Unfavorable
Mistral Large 4 itself returns `reasoning_content` with its tool calls, so a Mistral-only tool loop in OpenCode fails at round 2.
Mistral and InferencePort reject reasoning that agent clients send back. voodoohop, GitHub user, self-described, Oct 26, 2026. Mistral Large 4 on the Mistral API in an OpenCode tool loop.
- Favorable
tool loop: 6 of 6 runs called the tool correctly, with and without thinking.
feat: switch Ultra tier to Mistral Large 4. Movm, GitHub user and Gruenerator contributor, self-described, Oct 6, 2026. Mistral Large 4 (mistral-large-4) as Gruenerator's Ultra tier, tested live against the Mistral API on launch day.
- Unfavorable
Mistral's own models reject the history. A session that has used GLM fails on every request after a switch to `mistral/mistral-large-4-0`
Mistral-family tool-call ID normalization collapses GLM call IDs to "chatcmplt" (400 "Duplicate tool call id" on Mistral Large). infinitr0us, GitHub user, self-described, Oct 23, 2026. Mistral Large 4 (mistral-large-4-0) in OpenCode after a GLM session, through a gateway and directly on api.mistral.ai.
- Favorable
The Mistral API accepts an assistant message that carries both `thinking` content and `tool_calls` (verified on mistral-small-2603, mistral-medium-2604 and mistral-large-4-0)
langchain-mistralai: reasoning (thinking) blocks are dropped from assistant messages that carry tool calls. fchoron-pixel, GitHub user, self-described, Oct 19, 2026. Mistral Large 4 (mistral-large-4-0) through the Mistral API with thinking and tool calls in one assistant message.
- Favorable
Real OmO CLI continuation, high-thinking answer separation, and a successful tool roundtrip.
fix: support Mistral Large 4 thinking, continuations, and tool replay. audreyt, GitHub user, self-described, Oct 6, 2026. Mistral Large 4 in a real OmO CLI tool loop with high thinking.
- Favorable
the model works it out on its own through a series of tool calls: exploring the schema, writing queries, checking the results and writing up the answer
Mistral Large 4 Benchmark Results for Data Analysis. Plotly, company blog by Chris Parmer, co-founder and CPO, Oct 6, 2026. Mistral Large 4 in Plotly Studio Embedded on a 43-question agentic data analytics exam over a 20M-row warehouse.
- Unfavorable
Log excerpt from a real session (model: `mistral-large-4:cloud` via local Ollama daemon): ``` 11:35:10.721 WARN Malformed tool call JSON from provider
Malformed tool-call arguments replayed raw on next in-run request; strict providers (Ollama) 400 and kill the run. thylong, GitHub user, self-described, Oct 10, 2026. mistral-large-4:cloud through a local Ollama daemon in Crush.
- Unfavorable
`ollama-cloud/mistral-large-4` does this: each call has its own `id`, but `function.index` is `0` for all of them.
[Bug]: ollama-native fails turns where a model returns several tool calls at the same function index. MarcTCruz, GitHub user, self-described, Oct 9, 2026. ollama-cloud/mistral-large-4 from Codex CLI through opencodex.
- Favorable
Verified live Ollama Cloud mistral-large-4:cloud tool-calling (3.7s live test).
test: 5-pass self-proof loop, Ollama live testing & UX friction remediation. poferraz, GitHub user, self-described, Oct 16, 2026. mistral-large-4:cloud on Ollama Cloud as the agent model in allowagent-mcp.
- Unfavorable
Mistral Large 4 called a nonexistent bash 226 times in 209 replies ($4.33, 35% of its spend).
fix(agent): a 64k output budget, unknown-tool feedback, honest failed summaries, ordered headless JSON. gold-silver-copper, GitHub user and rig contributor, self-described, Oct 24, 2026. Mistral Large 4 as the agent model in the rig CLI through OpenRouter.
- Unfavorable
the model returned "label": ["cat", "dog"] and all five JSON parsers crashed
Add Mistral Large 4 under the full benchmark protocol. SkalskiP (Piotr Skalski), Roboflow vlm-exam maintainer, self-described, Oct 7, 2026. Mistral Large 4 through OpenRouter on Roboflow's vlm-exam, 6 vision tasks at low and high effort, 3 repeats.
Long context Needs 5 accounts from 3 authors
- Unfavorable
starts to forget instructions after awhile it seems
Mistral Large 4. delis-thumbs-7e, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through OpenRouter in a web UI, a half-hour chat on C and embedded programming.
- Favorable
Strong document and image reasoning: Mistral Large 4 Preview scores 19% on GDP.pdf
Mistral has released Mistral Large 4, making France home to the most intelligent model outside the US and China. Artificial Analysis, independent benchmark publisher, self-described, Oct 6, 2026. Mistral Large 4 Preview on the Artificial Analysis Intelligence Index, Cyber Index and GDP.pdf.
- Unfavorable
Mistral Large 4 gives OCR error when using document_url content
[BUG MODEL]: Mistral Large 4 gives OCR error when using document_url content. smathot, GitHub user, self-described, Oct 8, 2026. mistral-large-4 through the Mistral Python SDK with a PDF passed as document_url.
- Favorable
found every time) Lane Prompt tokens Time to first delta Ultra 97,493 17.3 s / 22.2 s Ultra 191,011 45.7 s Ultra 241,041 (56k cached) 35.3 s Ultra 861,094 111.2 s
feat(chat): give Ultra and Panda their measured ~1M context window. Movm, GitHub user and Gruenerator contributor, self-described, Oct 22, 2026. Mistral Large 4 (mistral-large-4) as Gruenerator's Ultra lane with prompts from 97k to 861k tokens on production keys.
- Unfavorable
11 end on the known Mistral limitation of images in tool results: get-plan-images overflows the context
docs: add the mistral-large-4 test results to the mistral backlog. Rabbid76, GitHub user and roomle-hi-example contributor, self-described, Oct 21, 2026. mistral-large-4 on 52 of 75 MCP planning tests against the roomle UI.
Developer experience 13%
- Unfavorable
it generated 200M tokens, which is very verbose in comparison to the median of 87M
Artificial Analysis - Mistral Large 4. Artificial Analysis, independent benchmark publisher, self-described, Oct 26, 2026. Mistral Large 4 (reasoning).
- Unfavorable
Very verbose and starts to forget instructions
Mistral Large 4. delis-thumbs-7e, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through OpenRouter in a web UI, a half-hour chat on C and embedded programming.
- Unfavorable
Surprisingly it only supports reasoning "none" or reasoning "high". That setting didn't seem to make any real difference
Mistral Large 4. simonw (Simon Willison), Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through the llm CLI at reasoning none and high, pelican SVG test.
- Unfavorable
reasoning effort parameter doesn't seem to be there and output is sort of broken because it reasons directly in the output tokens
Mistral Large 4. XCSme, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through the API on launch day.
- Unfavorable
The only ones available on my end are Medium 3.5 and GLM-5.3 despite having a Pro subscription.
Timestral: Mistral's time-series foundation models. pims, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 ("Mistral 4") access in Mistral Vibe on a Pro subscription.
- Unfavorable
multiple different sessions are convinced that they're in a simulation/fictional environment right off the bat
azziee on X: coding with Mistral Large 4. azzieewas, X user, self-described, Oct 21, 2026. Mistral Large 4 Preview used exclusively for coding.
- Unfavorable
please guys just make it an insufferable polytechnician instead of this suckup vizier
H. Gskrd on X: testing Mistral Large 4. h_gskrd, X user, self-described, Oct 14, 2026. Mistral Large 4, which the author calls GA and therefore usable with zero data retention; Mistral's model card lists it as a public preview, and Mistral's Commercial Terms (section 4.3) say zero data retention opt-outs do not apply to preview models.
- Unfavorable
it launched oct 6, it's oct 22 and there's no checkpoint to download
Duggal on X: no Mistral Large 4 checkpoint yet. Duggal, founder of darkfunnelhq, self-described, Oct 22, 2026. Mistral Large 4 open weights and list price on Oct 22.
- Favorable
improvements in their API, which now accepts 100 images per request, up from 8 for previous Mistral models
Mistral has released Mistral Large 4, making France home to the most intelligent model outside the US and China. Artificial Analysis, independent benchmark publisher, self-described, Oct 6, 2026. Mistral Large 4 Preview on the Artificial Analysis Intelligence Index, Cyber Index and GDP.pdf.
- Unfavorable
This blocks Codex clients that send developer-role instructions.
Cloud: mistral-large-4 rejects developer-role messages with HTTP 400. thericardoli, GitHub user, self-described, Oct 15, 2026. mistral-large-4 on Ollama Cloud through the Responses endpoint, called from Codex.
- Unfavorable
every call crashes with a TypeError, whether streaming is on or off
[BUG] MistralModel crashes on reasoning models (Mistral Large 4): content returned as list of ThinkChunk/TextChunk. margotmercier, GitHub user, self-described, Oct 9, 2026. Mistral Large 4 through Strands MistralModel with reasoning on.
- Unfavorable
Mistral provider shows a list of models but that list is not complete. For example, Mistral Large 4 and GLM 5.3 are missing.
Mistral provider doesn't work with certain Mistral models. tomsep, GitHub user, self-described, Oct 16, 2026. mistral-large-4 through the twinny VS Code extension's Mistral provider.
- Favorable
the issue is fixed for me with every thinking variants
Mistral Large 4 not working: type validation failed (invalid_union). Ronan-Le-Pivaingt-Dotscreen, GitHub user, self-described, Oct 9, 2026. opencode/mistral-large-4 with every thinking variant in OpenCode v2.0.26.
- Unfavorable
Large 4 accepts reasoning_effort: high and none , but rejects prompt_mode: reasoning .
fix: support Mistral Large 4 thinking, continuations, and tool replay. audreyt, GitHub user, self-described, Oct 6, 2026. Mistral Large 4 in a real OmO CLI tool loop with high thinking.
- Favorable
I updated my vibe config to point to Mistral 4 and it charges my subscription capacity, not my API allowance.
Timestral: Mistral's time-series foundation models. data-ottawa, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 ("Mistral 4") in Mistral Vibe through a config change on a subscription.
- Unfavorable
Mistral Large 4 (Le Chonk) higher reasoning effort scored worse and cost 8.3x more
SkalskiP on X: Mistral Large 4 reasoning effort. SkalskiP, open-source lead at Roboflow (X bio), self-described, Oct 12, 2026. Mistral Large 4 (Le Chonk) at low and high reasoning effort on his vision leaderboards.
- Unfavorable
Large 4 writes long structured answers: about 2–4K tokens even on tiny inputs
fix(providers): stream Mistral responses with an idle timeout instead of a 5-minute total deadline. akoita, GitHub user and draft-loop maintainer, self-described, Oct 8, 2026. Mistral Large 4 profile generation through the Mistral API without streaming.
- Unfavorable
Large 4 reasons by default, puts the reasoning inside content as parts, and reports 0 reasoning tokens in usage.
Roster v0.29: mistral-large-4-0 on the minimum-thinking channel (reasoning_effort none). srinivasgumdelli, GitHub user and seismograph contributor, self-described, Oct 10, 2026. Mistral Large 4 (mistral-large-4-0) on the Mistral API with default reasoning in the seismograph 87-call battery.
Cost 27%
- Favorable
well priced when comparing to other models of similar price
Artificial Analysis - Mistral Large 4. Artificial Analysis, independent benchmark publisher, self-described, Oct 26, 2026. Mistral Large 4 (reasoning).
- Favorable
The price is really good with ML4.
Mistral Large 4. pimeys, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 (ML4) pricing for summarization and content generation.
- Unfavorable
Xiaomi prices are still much better than the Mistral introductory offer for Le Chonk
Mistral Large 4. phillc73, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 (Le Chonk) introductory API pricing against Xiaomi MiMo-2.6.
- Favorable
It's 10x cheaper than Mistral Medium 3.5 from April
Mistral Large 4. chriddyp (Chris Parmer), co-founder of Plotly, Oct 6, 2026. Mistral Large 4 on Plotly's data analytics benchmark.
- Unfavorable
more expensive than both of those
Mistral Large 4. dom96, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 on the killswitch-lang programming language benchmark.
- Unfavorable
The cost is about 10x of Deepseek though comparing on OpenRouter
Mistral Large 4. Flux159, Hacker News commenter, self-described, Oct 21, 2026. Mistral Large 4 pricing on OpenRouter against DeepSeek 4.1.
- Unfavorable
Mistral Large 4 seems a much more expensive but overall weaker model across the board
Mistral Large 4. Systemerror7A69, Hacker News commenter, self-described, Oct 21, 2026. Mistral Large 4 against DeepSeek V4.1 Flash 1017 on price and benchmarks.
- Unfavorable
A proprietary model with performance comparable to GPT-6 Luna and Deepseek 4.1 Flash, but at a higher price than either
Mistral Large 4. wyrdcurt, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 against GPT-6 Luna and DeepSeek 4.1 Flash on price and performance.
- Unfavorable
This chonky model being expensive, mistral's subscription quota will run out ridiculously fast
Mistral Large 4. sbinnee, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 on a Mistral subscription.
- Unfavorable
Been using the mistral-large-4 now since it came out for a few things, and while it's not price competitive
Voxtral 3. barrell, Hacker News commenter, self-described, Oct 20, 2026. mistral-large-4 since launch, replacing browser chat use.
- Unfavorable
if you costed it during the launch weeks, your number is half the real one
OTHMANE M on X: mistral-large-4 price doubled. OTHMANEM9, X user who runs a price sync, self-described, Oct 25, 2026. mistral-large-4 API list price after the launch discount.
- Unfavorable
Over 4x the Cost per Task of similar-intelligence open weights models
Mistral has released Mistral Large 4, making France home to the most intelligent model outside the US and China. Artificial Analysis, independent benchmark publisher, self-described, Oct 6, 2026. Mistral Large 4 Preview on the Artificial Analysis Intelligence Index, Cyber Index and GDP.pdf.
- Favorable
This is very cost efficient and similar to models like OpenAI GPT-6.1 Sol ($2.61) and Qwen 3.8 27B ($1.71)
Mistral Large 4 Benchmark Results for Data Analysis. Plotly, company blog by Chris Parmer, co-founder and CPO, Oct 6, 2026. Mistral Large 4 in Plotly Studio Embedded on a 43-question agentic data analytics exam over a 20M-row warehouse.
- Unfavorable
However, it uses much more context on long agentic tasks, bringing its Vals Index cost to $13.78 per test.
Vals AI on X: Mistral Large 4 cost on the Vals Index. Vals AI, independent benchmark publisher, self-described, Oct 6, 2026. Mistral Large 4 on the Vals Index.
Speed 21%
- Favorable
At 113 tokens per second, Mistral Large 4 Preview is faster than average
Artificial Analysis - Mistral Large 4. Artificial Analysis, independent benchmark publisher, self-described, Oct 26, 2026. Mistral Large 4 (reasoning).
- Favorable
It is very fast via openrouter (significantly better than Kimi K3) on webui
Mistral Large 4. delis-thumbs-7e, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 through OpenRouter in a web UI, a half-hour chat on C and embedded programming.
- Unfavorable
GLM 5.3 (which Mistral also serves at 3x the speed!) wipes the floor with it
Mistral Large 4. TuxSH, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 running as subagents.
- Unfavorable
compared to GPT-6 luna it is much slower
Mistral Large 4. XCSme, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 against GPT-6 Luna on the aibenchy comparison.
- Unfavorable
it is much slower than Sonnet 5.5
Mistral Large 4. dom96, Hacker News commenter, self-described, Oct 7, 2026. Mistral Large 4 on the killswitch-lang programming language benchmark.
- Unfavorable
some days it's so slow I interrupt it because I thought it was hanging
Timestral: Mistral's time-series foundation models. sspiff, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 through Mistral's service over two weeks.
- Unfavorable
Just tested Mistral 4 too in Vibe and it's slow. It works well, but is way too slow.
Timestral: Mistral's time-series foundation models. maelito, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 ("Mistral 4") in Mistral Vibe.
- Unfavorable
It's slow as hell, and so is their GLM offering.
Timestral: Mistral's time-series foundation models. nozzlegear, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 ("Mistral 4") from OMP with a Vibe subscription.
- Favorable
Like basically instant responses that seem faster and better in quality than ChatGPT on instant mode
Mistral Large 4. bluerooibos, Hacker News commenter, self-described, Oct 6, 2026. Mistral Large 4 tried on launch day.
- Unfavorable
it is not a good fit for DwarfStar because of the very large number of active parameters, not just because of memory: would be too slow anyway
antirez on X: Mistral Large 4 and DwarfStar. antirez, programmer (X bio), self-described, Oct 21, 2026. Mistral Large 4 considered for local inference in DwarfStar.
- Favorable
It's also fast: it answers a typical question in about 22 seconds.
Mistral Large 4 Benchmark Results for Data Analysis. Plotly, company blog by Chris Parmer, co-founder and CPO, Oct 6, 2026. Mistral Large 4 in Plotly Studio Embedded on a 43-question agentic data analytics exam over a 20M-row warehouse.
- Unfavorable
One profile part therefore took more than 5 minutes, timed out, was retried, and failed again.
fix(providers): stream Mistral responses with an idle timeout instead of a 5-minute total deadline. akoita, GitHub user and draft-loop maintainer, self-described, Oct 8, 2026. Mistral Large 4 profile generation through the Mistral API without streaming.
- Unfavorable
our first-token idle deadline (20 s) fires while Mistral silently reads the prompt
feat(chat): give Ultra and Panda their measured ~1M context window. Movm, GitHub user and Gruenerator contributor, self-described, Oct 22, 2026. Mistral Large 4 (mistral-large-4) as Gruenerator's Ultra lane with prompts from 97k to 861k tokens on production keys.
- Unfavorable
because the model is too slow: about 3 minutes per run, the chat a median of 80 s and a mean of 153 s
docs: add the mistral-large-4 test results to the mistral backlog. Rabbid76, GitHub user and roomle-hi-example contributor, self-described, Oct 21, 2026. mistral-large-4 on 52 of 75 MCP planning tests against the roomle UI.
- Unfavorable
High-effort detection averages 161 s and 12k output tokens per image.
Add Mistral Large 4 under the full benchmark protocol. SkalskiP (Piotr Skalski), Roboflow vlm-exam maintainer, self-described, Oct 7, 2026. Mistral Large 4 through OpenRouter on Roboflow's vlm-exam, 6 vision tasks at low and high effort, 3 repeats.
Reliability 38%
- Unfavorable
I think they're shifting hosting capacity around, or otherwise messing with the model
Timestral: Mistral's time-series foundation models. sspiff, Hacker News commenter, self-described, Oct 19, 2026. Mistral Large 4 through Mistral's service over two weeks.
- Unfavorable
On launch day 3 of 4 image requests returned 503, so images keep going to VISION_MODEL .
feat: switch Ultra tier to Mistral Large 4. Movm, GitHub user and Gruenerator contributor, self-described, Oct 6, 2026. Mistral Large 4 (mistral-large-4) as Gruenerator's Ultra tier, tested live against the Mistral API on launch day.
- Unfavorable
a Mistral Large 4 profile run took 22 minutes, then saved a draft with **0 facts** and one issue: "The provider returned an invalid extraction response.
fix(profile): name the provider failure reason, and retry a Mistral part whose reply arrives broken. akoita, GitHub user and draft-loop maintainer, self-described, Oct 8, 2026. Mistral Large 4 profile extraction runs in draft-loop through the Mistral API.
- Unfavorable
During a Mistral API rate-limiting period (HTTP 429 responses), the CLI calls the identity endpoint every 2-4 s
bug: identity endpoint retried in a tight loop during Mistral 429s + silent CLI crash (CancelledError). Simon-LM, GitHub user, self-described, Oct 11, 2026. Mistral Large 4 (large-4) and GLM-5.3 in long-lived Mistral Vibe 2.26 sessions.
- Unfavorable
other/2026-10-06-mistral-mistral-large-4-rep-3-retry-1 TRANSPORT_ERROR (TRANSPORT)
Runs: mistral-large-4. Microck, GitHub user and keygen-bench maintainer, self-described, Oct 6, 2026. mistral-large-4 through the Mistral route on keygen-bench, three ranked attempts on launch day.
- Favorable
mistral-large-4-0@mistral answered 87/87 with plain-string content on reasoning_effort: none
Roster: mistral-large-4-0 and inkling confirmed live, drop availability=pending. srinivasgumdelli, GitHub user and seismograph contributor, self-described, Oct 10, 2026. Mistral Large 4 (mistral-large-4-0) on the Mistral API at reasoning_effort none in the seismograph 87-call battery.
- Favorable
All 13 run-quality checks and sensebench verify passed, with no transport errors or truncated responses.
Submit OpenRouter sweep: 13 runs across 9 models (p001). emilkayumov, GitHub user and sensebench contributor, self-described, Oct 13, 2026. Mistral Large 4 at reasoning none through OpenRouter in a 13-run sensebench sweep.
- Favorable
6 tasks x 2 efforts x 3 repeats, 36 result files, zero failed samples.
Add Mistral Large 4 under the full benchmark protocol. SkalskiP (Piotr Skalski), Roboflow vlm-exam maintainer, self-described, Oct 7, 2026. Mistral Large 4 through OpenRouter on Roboflow's vlm-exam, 6 vision tasks at low and high effort, 3 repeats.