Jev+40 Positive

Operator Insights

What are teams learning from building on Jev and running it in production?

20 accounts: 3 left it or turned it down, 1 switched, 9 evaluated it and 7 adopted it

Left it or turned it down3

srnsnemilkapa.ai team, posting its benchmarkkapa.ai
kapa.ai, which runs millions of retrieval queries a month over its customers' docs, Slack threads, tickets and code.
Reddit, r/LLMDevs, published Oct 20, 2026Read the write-up (opens Reddit, r/LLMDevs in a new tab)
Observation
kapa.ai found Jev 1.14 within 0.003 of the best reranker on 1,000 production-style cases, and chose Voyage instead because Voyage was also cheaper and faster.
Takeaway
Benchmark a reranker on your own queries: Jev nearly tied the leader on quality and still lost the decision on cost and speed.
What happened
kapa.ai reranked the top 50 hybrid-search chunks for 1,000 cases from its Company Knowledge Bench with Voyage, Cohere, Qwen and Jev. Jev 1.14 scored 0.532 against 0.535 for Voyage rerank-3, and beat Cohere rerank-v4.0-pro on score, cost (US$2.49 against US$3.42 per 1,000 questions) and median latency (0.28 against 0.77 seconds). Voyage rerank-3 also cost the least at US$1.91 and ran fastest at 0.18 seconds, so kapa.ai chose Voyage instead and is staying with it (Reddit, Oct 20, 2026).
- Public organization write-up (Reddit post)
- Measured results reported
Deployment details
kapa.ai, which runs millions of retrieval queries a month over its customers' docs, Slack threads, tickets and code.
- Public organization write-up (Reddit post)
- Measured results reported
Team
kapa.ai
Workload
Reranking 50 retrieved chunks for 1,000 Company Knowledge Bench cases
Product version
Jev 1.13 and Jev 1.14, binary and graded questions
Model
Voyage rerank-3 and rerank-2.5, Cohere rerank-v3.5 and v4.0, Qwen3 reranker 8B
Measurement
Retrieval score 0.532 for Jev 1.14 against 0.535 for Voyage rerank-3 (Oct 20, 2026)

Mike SwiftOct 8, 2026DEV CommunityMajor League Hacking got Jev to zero dangerous routing calls only after a prompt rewrite, and chose Gemini Flash-Lite instead for production.
Mike SwiftCEO and co-founderMajor League Hacking
A routing check at Major League Hacking, run a few thousand times on each model.
DEV Community, published Oct 8, 2026Read the write-up (opens DEV Community in a new tab)
Observation
Major League Hacking got Jev to zero dangerous routing calls only after a prompt rewrite, and chose Gemini Flash-Lite instead for production.
Takeaway
Compare dangerous-error rates on your original prompt, and price Jev as a first-stage filter in front of the model you already trust.
What happened
MLH compared Jev with Gemini Flash-Lite across thousands of routing runs. Flash-Lite made no dangerous calls on the original prompt, and Jev reached zero only after a prompt rewrite. On a public function-calling slice Jev scored 94.6% label accuracy at 261 ms and US$0.025 per 1,000 calls, against 96.9% at 800 ms and US$0.257 for Flash-Lite, which also writes each call's arguments (DEV, Oct 8, 2026). MLH chose Flash-Lite instead for production, and found that pairing the two cost US$0.10 per 1,000 checks against US$0.52 for Flash-Lite alone.
Public practitioner write-up
Measured results reported
Deployment details
A routing check at Major League Hacking, run a few thousand times on each model.
Public practitioner write-up
Measured results reported
Team
Major League Hacking
Workload
Routing checks over a few thousand runs
Model
Jev against Gemini Flash-Lite
Measurement
94.6% at 261 ms and US$0.025 per 1,000 calls against 96.9% at 800 ms and US$0.257 (Oct 8, 2026)
nimSep 22, 2026Xdessn dropped more than 10,000 Jev calls for a single Claude Opus 5.5 call that costs a little more and is more accurate.
nimCo-founderdessn
dessn, a startup building design tooling that works inside a team's production codebase.
X, published Sep 22, 2026Read the write-up (opens X in a new tab)
Observation
dessn dropped more than 10,000 Jev calls for a single Claude Opus 5.5 call that costs a little more and is more accurate.
Takeaway
Price the single large-model call beside the many-decision pipeline; when one model can see the whole task, it can win on accuracy at a similar cost.
What happened
dessn dropped more than 10,000 Jev calls, along with a few small models, and moved that work to one Claude Opus 5.5 call (X, Sep 22, 2026). Its co-founder reports the single call costs a bit more and runs slightly slower, with significant accuracy gains.
“We replaced over 10,000 jev calls and a few fly brains with 1 opus 5.5 call.”
nim
Public practitioner write-up (X post)
Cost and accuracy reported without figures
Deployment details
dessn, a startup building design tooling that works inside a team's production codebase.
Public practitioner write-up (X post)
Cost and accuracy reported without figures
Team
dessn
Workload
More than 10,000 decision calls in dessn's product
Model
Claude Opus 5.5 in place of Jev

Switched to or from it1

Matthew WilsonCo-founder and CEOJack & Jill
Jack & Jill, an AI recruiting company, in one stage of its candidate-matching pipeline.
TypeSafe, published Oct 7, 2026Read the report (opens TypeSafe in a new tab)
Observation
Jack & Jill replaced Gemini 3.1 Flash Lite with Jev for one candidate-matching stage within 10 days of its first test.
Takeaway
Read the method note before quoting a vendor's saving: the annual projection at current volume was about half the growth-based headline projection.
What happened
Jack & Jill replaced Gemini 3.1 Flash Lite with Jev for 100% of calls in a key stage of its candidate-matching pipeline within 10 days of its first test, TypeSafe's case study says (Oct 7, 2026). The US$500K saving in the headline is a projection on expected search growth; the case study puts savings at current volume at US$265K, and its quality figures come from an offline test on live roles.
“We saved ~half a million dollars last week with Jev. And our products got much better at the same time. Matthew Wilson, Co-founder/CEO @ Jack & Jill…”
Matthew Wilson
- Vendor customer story (TypeSafe)
- Projected saving, with its basis stated
Deployment details
Jack & Jill, an AI recruiting company, in one stage of its candidate-matching pipeline.
- Vendor customer story (TypeSafe)
- Projected saving, with its basis stated
Team
Jack & Jill
Duration
10 days from first test to full rollout
Workload
One stage of candidate matching, 100% of its calls
Model
Jev in place of Gemini 3.1 Flash Lite
Measurement
US$265K a year projected at current volume; US$500K projected on growth (Oct 7, 2026)

Evaluated it9

Eleanor SigelAuthor, Casco benchmarkCasco
Casco, a penetration-testing company, scoring findings for CVSS 3.1 severity.
Its own announcement, published Sep 28, 2026Read the write-up (opens the announcement in a new tab)
Observation
Casco tested Jev on 2,449 security findings and got 550 critical scores where its reference held 55.
Takeaway
Keep a person on severity calls: a typed answer can be well-formed and still inflate risk.
What happened
Casco tested eight models, Jev, Claude and GPT among them, on 2,449 findings against its published CVSS 3.1 scores, giving each the same evidence without Casco's scoring policy (Sep 28, 2026). Jev assigned 550 critical scores to a set holding 55 published critical findings, and 500 of those 550 were below critical in the reference. Every model scored higher than the reference on average; GPT-6 Astra did best overall.
- Public engineering write-up
- Measured results reported
Deployment details
Casco, a penetration-testing company, scoring findings for CVSS 3.1 severity.
- Public engineering write-up
- Measured results reported
Team
Casco
Workload
CVSS 3.1 scoring of 2,449 security findings
Model
Jev beside seven other models, Claude and GPT among them
Measurement
550 critical scores where the reference held 55 critical findings (Sep 28, 2026)

CiscoOct 5, 2026Cisco BlogsCisco's AI Defense team found a 1B encoder trained on its own policy beat zero-shot Jev on all six safety datasets.
Prashanth ArunDirector of Data Science, AI DefenseCisco
Cisco AI Defense's content-safety evaluation on six safety evaluation datasets and eight public benchmarks.
Cisco Blogs, published Oct 5, 2026Read the write-up (opens Cisco Blogs in a new tab)
Observation
Cisco's AI Defense team found a 1B encoder trained on its own policy beat zero-shot Jev on all six safety datasets.
Takeaway
Where a policy and labeled data already exist, test a small trained classifier before routing safety decisions to Jev.
What happened
Cisco's AI Defense team evaluated zero-shot Jev for content safety. At a 0.5% false-positive budget, a 1B encoder trained on Cisco's policy beat Jev on all six datasets, with recall of 61% against 34% on Cisco multi-turn. Jev's AUC beat a prompted Gemma 4 31B judge on seven of eight public benchmarks (Cisco Blogs, Oct 5, 2026).
Public engineering write-up
Measured results reported
Deployment details
Cisco AI Defense's content-safety evaluation on six safety evaluation datasets and eight public benchmarks.
Public engineering write-up
Measured results reported
Team
Cisco AI Defense
Workload
Content-safety classification on six datasets and eight public benchmarks
Model
Jev against a 1B trained encoder and a prompted Gemma 4 31B judge
Measurement
Recall 61% against 34% on Cisco multi-turn at a 0.5% false-positive budget (Oct 5, 2026)
AnacondaOct 20, 2026AnacondaAnaconda tested zero-shot Jev on 2,400 consumer complaints, and it trailed a most-common-answer baseline on every question.
Philipp RudigerAuthor, Anaconda Technical NotesAnaconda
Anaconda's technical note comparing Jev with a fine-tuned classifier on held-out consumer complaints.
Its own announcement, published Oct 20, 2026Read the write-up (opens the announcement in a new tab)
Observation
Anaconda tested zero-shot Jev on 2,400 consumer complaints, and it trailed a most-common-answer baseline on every question.
Takeaway
With a few thousand labels in hand, train the classifier first and treat Jev as the baseline to beat.
What happened
Anaconda tested Jev on 2,400 held-out consumer complaints against its own fine-tuned small model and against XGBoost models trained on each complaint's structured fields. Asked cold, Jev scored below always guessing the most common answer on all three questions; with the labeling rules written into the prompt it reached 82% on quick review and 54% on severity, still below the XGBoost model that never reads the complaint. On the resolution question its adjusted probabilities matched how often it was right, at about US$0.00003 per decision (Oct 20, 2026).
Public engineering write-up
Measured results reported
Deployment details
Anaconda's technical note comparing Jev with a fine-tuned classifier on held-out consumer complaints.
Public engineering write-up
Measured results reported
Team
Anaconda
Workload
Classifying 2,400 held-out consumer complaints
Model
Jev against Anaconda's fine-tuned model and XGBoost baselines
Measurement
Below a most-common-answer baseline on all three questions; about US$0.00003 per decision (Oct 20, 2026)
Southbridge AISep 19, 2026Southbridge AISouthbridge tested Jev as an observer on its agent runs: forecast error fell 17.84% for US$2.30, and attacks found 15 bypasses of its command gate.
Hrishi OlickelAuthor, Southbridge AI engineering write-upSouthbridge AI
Southbridge's Hankweave agent runs: 220,000 tool calls and 500 replayed runs.
Its own announcement, published Sep 19, 2026Read the write-up (opens the announcement in a new tab)
Observation
Southbridge tested Jev as an observer on its agent runs: forecast error fell 17.84% for US$2.30, and attacks found 15 bypasses of its command gate.
Takeaway
Jev is a cheap way to watch agent runs; do not make it the only guard on harmful commands.
What happened
Southbridge tested Jev as an observer on its agent runs. Adding Jev's readings cut held-out forecast error for remaining time by 17.84% for US$2.30 of usage, and adaptive attacks found 15 bypasses of its harmful-command gate (Sep 19, 2026).
Public engineering write-up
Measured results reported
Deployment details
Southbridge's Hankweave agent runs: 220,000 tool calls and 500 replayed runs.
Public engineering write-up
Measured results reported
Team
Southbridge AI
Workload
Observing agent runs across 220,000 tool calls and 500 replayed runs
Measurement
17.84% lower forecast error for US$2.30; 15 gate bypasses found (Sep 19, 2026)
Drew DillonSep 17, 2026XBrief benchmarked Jev against its production judges: one sales-call classification test ran about 100 times cheaper per call, and a code-review test found none of 13 known problems.
Drew DillonCEO and co-founderBrief
Brief's production LLM judges, replayed against Jev over about 3,700 calls.
X, published Sep 17, 2026Read the write-up (opens X in a new tab)
Observation
Brief benchmarked Jev against its production judges: one sales-call classification test ran about 100 times cheaper per call, and a code-review test found none of 13 known problems.
Takeaway
Move fixed-label checks to Jev and keep rubric and review judgments on the model that already passes them.
What happened
Brief benchmarked Jev against its production judges over about 3,700 calls, with nine server errors and a 100 to 200 ms median. On fixed-label tasks it matched or beat the models Brief runs today: it classified 35 of 36 sales-call transcripts at about 100 times lower cost per call. On judgment tasks it failed: a Jev-only reviewer on 14 merged pull requests found none of the 13 problems people had found in code review (X, Sep 17, 2026).
Public practitioner write-up (X post)
Measured results reported
Deployment details
Brief's production LLM judges, replayed against Jev over about 3,700 calls.
Public practitioner write-up (X post)
Measured results reported
Team
Brief
Workload
Production judges for fixed-label tasks, rubric grading and code review
Model
Jev against Brief's production LLM judges
Measurement
About 3,700 calls, nine server errors, 0 of 13 code-review problems found (Sep 17, 2026)
Guillermo RauchSep 16, 2026XVercel's fx team benchmarked Jev as the safety reviewer for every auto-mode command and found it up to 18 times faster at p95.
Guillermo RauchCEOVercel
fx, Vercel's command-line agent, where a reviewer model checks every command in auto mode.
X, published Sep 16, 2026Read the write-up (opens X in a new tab)
Observation
Vercel's fx team benchmarked Jev as the safety reviewer for every auto-mode command and found it up to 18 times faster at p95.
Takeaway
A per-command safety check is a strong fit when latency sits in the user's loop; measure accuracy on your own command log before switching.
What happened
Vercel's fx team benchmarked its auto-mode safety reviewer on Jev. Rauch reports Jev up to 18 times faster at p95 than GPT Luna, the team's top choice at the time, and more accurate, and expects it to become the default (X, Sep 16, 2026).
Public practitioner write-up (X post)
Speed reported
Deployment details
fx, Vercel's command-line agent, where a reviewer model checks every command in auto mode.
Public practitioner write-up (X post)
Speed reported
Team
Vercel fx team
Workload
Safety review of every command in fx auto mode
Model
Jev against GPT Luna
Measurement
Up to 18 times faster at p95 (Sep 16, 2026)
Alex BouazizSep 25, 2026XDeel tested Jev against frontier LLMs: up to 59 times cheaper, 4 times faster, and level or better on 6 of 8 quality checks.
Alex BouazizCEO and co-founderDeel
Deel, the global payroll and HR company, checking Jev against frontier LLMs on its own tasks.
X, published Sep 25, 2026Read the write-up (opens X in a new tab)
Observation
Deel tested Jev against frontier LLMs: up to 59 times cheaper, 4 times faster, and level or better on 6 of 8 quality checks.
Takeaway
Test Jev on your own repeat questions and categories: Deel reported repeat-question accuracy improving from 70% to 97%.
What happened
Deel tested Jev against frontier LLMs and its CEO reports it up to 59 times cheaper and 4 times faster, with quality at parity or better on 6 of 8 checks. Repeat-question accuracy rose from 70% to 97% and expense categorization from 50% to 86% (X, Sep 25, 2026).
Public practitioner write-up (X post)
Measured results reported
Deployment details
Deel, the global payroll and HR company, checking Jev against frontier LLMs on its own tasks.
Public practitioner write-up (X post)
Measured results reported
Team
Deel
Model
Jev against frontier LLMs
Measurement
Up to 59 times cheaper and 4 times faster; 6 of 8 checks at parity or better (Sep 25, 2026)
SanchitSep 27, 2026XA Spinny engineer tested Jev against the team's production OpenAI ticket triage and found it about 9 times faster and 10 times cheaper.
SanchitSoftware engineer, Customer Experience teamSpinny
Spinny's customer-experience ticket categorization and urgency detection.
X, published Sep 27, 2026Read the write-up (opens X in a new tab)
Observation
A Spinny engineer tested Jev against the team's production OpenAI ticket triage and found it about 9 times faster and 10 times cheaper.
Takeaway
Ticket triage with fixed categories is where Jev's speed and price show first; compare label agreement before switching.
What happened
A Spinny engineer tested Jev against the team's production OpenAI pipeline for ticket categorization and urgency detection, and found Jev about 9 times faster and 10 times cheaper (X, Sep 27, 2026).
Public practitioner write-up (X post)
Speed and cost reported
Deployment details
Spinny's customer-experience ticket categorization and urgency detection.
Public practitioner write-up (X post)
Speed and cost reported
Team
Spinny Customer Experience
Workload
Ticket categorization and urgency detection
Model
Jev against the production OpenAI model
Measurement
About 9 times faster and 10 times cheaper (Sep 27, 2026)
Nikhil MudholkarSep 17, 2026XGemini beat Jev on Bryo's benchmark of 1,565 supplier emails, and Bryo's CTO still wants Jev in production because of where its mistakes fell.
Nikhil MudholkarCTO and co-founderBryo
Bryo's benchmark of 1,565 business emails from industrial suppliers.
X, published Sep 17, 2026Read the write-up (opens X in a new tab)
Observation
Gemini beat Jev on Bryo's benchmark of 1,565 supplier emails, and Bryo's CTO still wants Jev in production because of where its mistakes fell.
Takeaway
Read the errors as well as the score: a lower-scoring model can still fail on cases that cost less.
What happened
Bryo benchmarked Jev against Gemini on 1,565 business emails from industrial suppliers, and Gemini scored higher. Its CTO still wants to put Jev into production because of where its mistakes fell (X, Sep 17, 2026).
Public practitioner write-up (X post)
Measured results reported
Deployment details
Bryo's benchmark of 1,565 business emails from industrial suppliers.
Public practitioner write-up (X post)
Measured results reported
Team
Bryo
Workload
Classifying 1,565 supplier emails
Model
Jev against Gemini

Adopted it7

VectorizeSep 24, 2026Hindsight blog (Vectorize)Vectorize shipped Jev as Hindsight's reranker at recall@1 of 0.950 against 0.800, and left its pruning off because it cut 19% of the evidence.
Ben BartholomewHindsight teamVectorize
Hindsight, Vectorize's open-source memory server for agents, release 0.10.1.
Hindsight blog (Vectorize), published Sep 24, 2026Read the write-up (opens Hindsight blog (Vectorize) in a new tab)
Observation
Vectorize shipped Jev as Hindsight's reranker at recall@1 of 0.950 against 0.800, and left its pruning off because it cut 19% of the evidence.
Takeaway
Use Jev's ranking and keep pruning off until you have measured what it drops.
What happened
Every Hindsight deployment can now run Jev as an opt-in reranker. On LoCoMo with 30 candidates, the team measured recall@1 of 0.950 for Jev against 0.800 for its default local MiniLM reranker, at 0.027 against 0.12 seconds per query (Sep 24, 2026). Optional pruning cut 19% of the gold evidence, so it ships off.
“It also cuts 19% of the gold evidence. On a real bank it took 300 candidates down to 3. We went back and forth on the default and landed on off.”
Ben Bartholomew
Public engineering write-up
Measured results reported
Deployment details
Hindsight, Vectorize's open-source memory server for agents, release 0.10.1.
Public engineering write-up
Measured results reported
Team
Hindsight core team at Vectorize
Workload
Reranking memories at recall time
Product version
Hindsight 0.10.1 with Jev
Model
Jev against a local MiniLM reranker
Measurement
Recall@1 0.950 against 0.800 at 0.027 against 0.12 seconds per query (Sep 24, 2026)
MotherDuckSep 21, 2026MotherDuckMotherDuck built prompt_jev() into SQL after Jev matched frontier accuracy on 100,000 rows in 40 seconds for fifty cents.
Hamilton Ulmer, Till Döhmen and Jacob MatsonAuthors of MotherDuck's launch postMotherDuck
MotherDuck, a hosted DuckDB data warehouse, adding text classification to SQL on its paid plans.
Its own announcement, published Sep 21, 2026Read the write-up (opens the announcement in a new tab)
Observation
MotherDuck built prompt_jev() into SQL after Jev matched frontier accuracy on 100,000 rows in 40 seconds for fifty cents.
Takeaway
For bulk labeling inside a warehouse, test Jev against the LLM function you already call: MotherDuck's 100,000-row benchmark cost US$0.50 on Jev against US$37.58 on the comparable LLM.
What happened
MotherDuck built prompt_jev() so customers can classify text in SQL on every paid plan. On 100,000 AG News articles, its benchmark had Jev match frontier LLM accuracy in 40 seconds for fifty cents, against more than half an hour and US$37 for the comparable LLM (Sep 21, 2026).
“Today we're shipping an integration with Jev, a new kind of AI model from TypeSafe AI. On a 100,000-row benchmark it matched frontier LLM accuracy in 40 seconds for fifty cents. The comparable LLM took more than half an hour and cost $37.”
Hamilton Ulmer, Till Döhmen and Jacob Matson
Public engineering write-up
Measured results reported
Deployment details
MotherDuck, a hosted DuckDB data warehouse, adding text classification to SQL on its paid plans.
Public engineering write-up
Measured results reported
Team
MotherDuck
Workload
Text classification in SQL, benchmarked on 100,000 AG News articles
Model
Jev against a frontier LLM
Measurement
40 seconds and US$0.50 against more than 30 minutes and US$37 (Sep 21, 2026)
Browser UseSep 17, 2026GitHubBrowser Use built jev-ultrafast, a web agent where Jev picks every action, and cut one task's median time from 9.45 to 7.09 seconds.
Browser UseRepository ownerBrowser Use
Browser Use's open-source web agent, with a waitlist for a hosted version.
GitHub, published Sep 17, 2026Read the write-up (opens GitHub in a new tab)
Observation
Browser Use built jev-ultrafast, a web agent where Jev picks every action, and cut one task's median time from 9.45 to 7.09 seconds.
Takeaway
Let Jev choose browser actions and call a writer model only to type; test on more than one task before trusting the speedup.
What happened
Browser Use built jev-ultrafast, an open-source web agent branded Browser Use × TypeSafe, in which Jev picks each operation and target element and a small LLM writes text only when typing is needed (GitHub, Sep 17, 2026). Its own test on one Google Flights task, three runs per version, cut the median task time from 9.450 to 7.092 seconds.
“Median task time went from **9.450 s → 7.092 s**, a **25% reduction**; median browser protocol calls went from **1,092 → 101**.”
Browser Use
Public engineering write-up (GitHub README)
Measured results reported
Deployment details
Browser Use's open-source web agent, with a waitlist for a hosted version.
Public engineering write-up (GitHub README)
Measured results reported
Team
Browser Use
Workload
Web agent actions, measured on one Google Flights task
Model
Jev for actions with a small LLM for typing
Measurement
Median 9.450 to 7.092 seconds over three runs per version (Sep 17, 2026)
Isotopes AIOct 15, 2026Isotopes AIIsotopes AI runs Jev 1.13 as its production default after it passed 100% across three runs of 88 scenarios, and held back the Jev 1.14 pre-release that passed 93.6%.
Prasanth JayachandranAuthor, Isotopes AI engineering write-upIsotopes AI
Isotopes AI's follow-up workflow in its enterprise data agent, an 88-scenario suite run three times per model.
Its own announcement, published Oct 15, 2026Read the write-up (opens the announcement in a new tab)
Observation
Isotopes AI runs Jev 1.13 as its production default after it passed 100% across three runs of 88 scenarios, and held back the Jev 1.14 pre-release that passed 93.6%.
Takeaway
Pin the Jev version you measured and rerun the suite before an upgrade, because a newer build scored lower on this workflow.
What happened
Across three runs of its 88-scenario workflow suite, Jev 1.13 passed 100% at a 0.807-second median, against 68.2% and 1.546 seconds for OpenAI Decisions (Isotopes AI, Oct 15, 2026). The Jev 1.14 pre-release build passed 93.6% on the same suite, so Isotopes AI runs Jev 1.13 as the production default.
Public engineering write-up
Measured results reported
Deployment details
Isotopes AI's follow-up workflow in its enterprise data agent, an 88-scenario suite run three times per model.
Public engineering write-up
Measured results reported
Team
Isotopes AI
Workload
88-scenario follow-up workflow, three runs per model
Product version
Jev 1.13 in production; Jev 1.14 pre-release tested
Model
Jev 1.13 against OpenAI Decisions
Measurement
100% passed across three runs of 88 scenarios at a 0.807-second median; 93.6% for the Jev 1.14 pre-release (Oct 15, 2026)
ServerpodSep 23, 2026ServerpodServerpod uses Jev to pick documents for its Starguide assistant, and time to the first streamed token roughly halved.
Viktor LidholtFounderServerpod
Starguide, the production assistant that answers questions about the Serverpod framework.
Its own announcement, published Sep 23, 2026Read the write-up (opens the announcement in a new tab)
Observation
Serverpod uses Jev to pick documents for its Starguide assistant, and time to the first streamed token roughly halved.
Takeaway
Put Jev in front of retrieval to choose documents, and keep the slower search as the fallback.
What happened
Starguide used to normalize each question with an LLM and search a pgvector index before writing an answer. Serverpod now uses Jev to pick the relevant documentation pages first and falls back to the full search when nothing fits, and time to the first streamed token roughly halved (Sep 23, 2026). Jev also checks whether each question was answered, which points the team at gaps in its documentation.
Public engineering write-up
Latency change reported
Deployment details
Starguide, the production assistant that answers questions about the Serverpod framework.
Public engineering write-up
Latency change reported
Team
Serverpod
Workload
Document choice for Starguide, Serverpod's documentation assistant
Model
Jev in front of an LLM writer and a gemini-embedding index
Measurement
Time to first streamed token roughly halved (Sep 23, 2026)
ChatPRDSep 28, 2026How I AIChatPRD's founder runs Jev across five projects, including a product-insights graph built from 200,000 classifications.
Claire VoFounderChatPRD
ChatPRD and the host's own projects, shown on the How I AI podcast.
How I AI, published Sep 28, 2026Listen to the episode (opens How I AI in a new tab)
Observation
ChatPRD's founder runs Jev across five projects, including a product-insights graph built from 200,000 classifications.
Takeaway
Bulk sorting of a backlog is the cheapest first test: 1,700 items cost 9 cents in this episode.
What happened
On How I AI, Claire Vo runs Jev on five of her own projects. It sorted 1,700 pull requests for 9 cents, and it built ChatPRD's product-insights graph from 1,100 signals and 200,000 classifications (Sep 28, 2026).
Podcast episode (How I AI)
Cost reported
Deployment details
ChatPRD and the host's own projects, shown on the How I AI podcast.
Podcast episode (How I AI)
Cost reported
Team
ChatPRD
Workload
Pull-request sorting, email triage and a product-insights graph
Model
Jev, with an LLM for follow-up writing
Measurement
1,700 pull requests for 9 cents; 200,000 classifications for the insights graph (Sep 28, 2026)
mlahargouOct 2, 2026GitHubiFixit runs Jev in FixBot and has to script the logged-in console to track daily spend, because Jev's API has no usage endpoint.
mlahargouEngineer on FixBotiFixit
FixBot, iFixit's repair assistant, with a daily cost sheet for every AI service the team pays for.
GitHub, published Oct 2, 2026Read the write-up (opens GitHub in a new tab)
Observation
iFixit runs Jev in FixBot and has to script the logged-in console to track daily spend, because Jev's API has no usage endpoint.
Takeaway
Plan cost tracking before rollout: Jev reports usage only in its console, with no endpoint an API key can read.
What happened
iFixit runs Jev in FixBot and fills a daily cost sheet for every AI service it pays for. OpenAI, Google and Vercel let the team fetch usage with an API key; for Jev it has to script the logged-in console, because the API offers no usage endpoint (GitHub, Oct 2, 2026).
Public practitioner write-up (GitHub issue)
Deployment details
FixBot, iFixit's repair assistant, with a daily cost sheet for every AI service the team pays for.
Public practitioner write-up (GitHub issue)
Team
iFixit FixBot team
Workload
Decisions inside FixBot, iFixit's repair assistant

Left it or turned it down3

kapa.ai found Jev 1.14 within 0.003 of the best reranker on 1,000 production-style cases, and chose Voyage instead because Voyage was also cheaper and faster.

Major League Hacking got Jev to zero dangerous routing calls only after a prompt rewrite, and chose Gemini Flash-Lite instead for production.

dessn dropped more than 10,000 Jev calls for a single Claude Opus 5.5 call that costs a little more and is more accurate.

Switched to or from it1

Jack & Jill replaced Gemini 3.1 Flash Lite with Jev for one candidate-matching stage within 10 days of its first test.

Evaluated it9

Casco tested Jev on 2,449 security findings and got 550 critical scores where its reference held 55.

Cisco's AI Defense team found a 1B encoder trained on its own policy beat zero-shot Jev on all six safety datasets.

Anaconda tested zero-shot Jev on 2,400 consumer complaints, and it trailed a most-common-answer baseline on every question.

Southbridge tested Jev as an observer on its agent runs: forecast error fell 17.84% for US$2.30, and attacks found 15 bypasses of its command gate.

Brief benchmarked Jev against its production judges: one sales-call classification test ran about 100 times cheaper per call, and a code-review test found none of 13 known problems.

Vercel's fx team benchmarked Jev as the safety reviewer for every auto-mode command and found it up to 18 times faster at p95.

Deel tested Jev against frontier LLMs: up to 59 times cheaper, 4 times faster, and level or better on 6 of 8 quality checks.

A Spinny engineer tested Jev against the team's production OpenAI ticket triage and found it about 9 times faster and 10 times cheaper.

Gemini beat Jev on Bryo's benchmark of 1,565 supplier emails, and Bryo's CTO still wants Jev in production because of where its mistakes fell.

Adopted it7

Vectorize shipped Jev as Hindsight's reranker at recall@1 of 0.950 against 0.800, and left its pruning off because it cut 19% of the evidence.

MotherDuck built prompt_jev() into SQL after Jev matched frontier accuracy on 100,000 rows in 40 seconds for fifty cents.

Browser Use built jev-ultrafast, a web agent where Jev picks every action, and cut one task's median time from 9.45 to 7.09 seconds.

Isotopes AI runs Jev 1.13 as its production default after it passed 100% across three runs of 88 scenarios, and held back the Jev 1.14 pre-release that passed 93.6%.

Serverpod uses Jev to pick documents for its Starguide assistant, and time to the first streamed token roughly halved.

ChatPRD's founder runs Jev across five projects, including a product-insights graph built from 200,000 classifications.

iFixit runs Jev in FixBot and has to script the logged-in console to track daily spend, because Jev's API has no usage endpoint.