Fireworks AI−17 Mixed

Operator Insights

What are teams learning from building on Fireworks AI and running it in production?

3 accounts: 1 failure, incident or rollback and 2 adopted it

Failures, incidents and rollbacks1

Omar QunsulSenior Backend Engineer, AI Coding, at the time of the changeGitLab
GitLab's IDE code completions in the asia-northeast1 and us-east4 regions, served from on-demand Codestral deployments on Fireworks through GitLab's AI gateway. GitLab lists 2,500+ team members (June 2026).
Its own announcement, published Mar 24, 2026Read the merge request (opens the announcement in a new tab)
Observation
GitLab's incident record traced a 12.03% code-completion error rate to Fireworks being unable to scale a new Codestral 2508 deployment, and GitLab redirected the traffic to Codestral 2501.
Takeaway
Before a newly launched model becomes a production default, reserve capacity for it and keep a tested redirect to the previous deployment.
What happened
On March 24, 2026 GitLab's incident record INC-8707 (Severity 4) reported a 12.03% error rate for code completions in asia-northeast1 and us-east4, and gave the cause as Fireworks being unable to scale up a deployment for the new Codestral 2508 model. GitLab was on on-demand 2508 deployments that used Fireworks' legacy direct routing. Omar Qunsul's merge request redirected Codestral 2508 requests to the older Codestral 2501 deployment, and the incident's 12:42 UTC update said no drop in error rates had been observed yet. GitLab then moved to reserved deployments behind Fireworks' standard API gateway, merged at 19:09 UTC the same day.
- Public engineering write-up (merge request and incident record)
- Measured results reported
Deployment details
GitLab's IDE code completions in the asia-northeast1 and us-east4 regions, served from on-demand Codestral deployments on Fireworks through GitLab's AI gateway. GitLab lists 2,500+ team members (June 2026).
- Public engineering write-up (merge request and incident record)
- Measured results reported
Team
GitLab AI Coding
Duration
March 24, 2026
Workload
IDE code completions in asia-northeast1 and us-east4
Product version
On-demand deployments with legacy direct routing, then reserved deployments behind the standard API gateway
Model
Codestral 2508, redirected to Codestral 2501
Measurement
12.03% code-completion error rate, Severity 4, in GitLab's incident record INC-8707 (Mar 24, 2026)

Adopted it2

Aman SangerCo-founderCursor
Cursor (Anysphere), maker of the Cursor AI coding agent and development environment, about 300 employees in 2025, applying planned code edits to whole files with its own fine-tuned 70b model in May 2024.
Cursor blog, published May 14, 2024Read the write-up (opens Cursor blog in a new tab)
Observation
Cursor deployed its fine-tuned 70b fast-apply model with Fireworks, which built API support for Cursor's speculative edits, and reports about 1000 tokens per second.
Takeaway
When a latency-critical feature runs on your own fine-tuned model, ask a host whether it will support your decoding method in its API, since Cursor's speedup came from that method.
What happened
In May 2024 Cursor deployed its own fast-apply model, a fine-tuned 70b model for full-file code edits, with Fireworks, and Fireworks built API support for Cursor's custom speculation logic. Cursor reports about 1000 tokens per second, about 13x faster than vanilla inference using Llama-3-70b, and credits the gain to its own speculative edits method.
- Public engineering write-up
- Measured results reported
Deployment details
Cursor (Anysphere), maker of the Cursor AI coding agent and development environment, about 300 employees in 2025, applying planned code edits to whole files with its own fine-tuned 70b model in May 2024.
- Public engineering write-up
- Measured results reported
Team
Cursor (Anysphere), about 300 employees in 2025
Workload
Full-file code edits from Cursor's fast-apply model
Model
Cursor's own fine-tuned 70b fast-apply model
Measurement
About 1000 tokens (about 3500 characters) per second, about 13x faster than vanilla Llama-3-70b inference (May 14, 2024)

SourcegraphDec 14, 2023Latent SpaceSourcegraph runs StarCoder for most Cody community-edition completions on Fireworks, so the team does not build its own inference stack.
Beyang LiuCo-founderSourcegraph
Sourcegraph's Cody code completions in the community edition, 2023.
Latent Space, published Dec 14, 2023Read the episode (opens Latent Space in a new tab)
Observation
Sourcegraph runs StarCoder for most Cody community-edition completions on Fireworks, so the team does not build its own inference stack.
Takeaway
Buy serving when your edge is context and fine-tuning, and credit a latency gain to the model change or the host only after you measure each.
What happened
In 2023 Sourcegraph moved most Cody community-edition completions to StarCoder served on Fireworks. Co-founder Beyang Liu said on Latent Space on December 14, 2023 that running StarCoder on Fireworks meant the team did not have to build its own inference stack and could focus on data fetching, the knowledge graph and model fine-tuning. Sourcegraph's October 2023 release post credits its autocomplete latency gains to incorporating open source LLMs and leaves Fireworks uncredited.
Podcast episode
No metrics credited to the host
Deployment details
Sourcegraph's Cody code completions in the community edition, 2023.
Podcast episode
No metrics credited to the host
Team
Sourcegraph (Cody)
Workload
Cody community-edition code completions
Model
StarCoder

Failures, incidents and rollbacks1

GitLab's incident record traced a 12.03% code-completion error rate to Fireworks being unable to scale a new Codestral 2508 deployment, and GitLab redirected the traffic to Codestral 2501.

Adopted it2

Cursor deployed its fine-tuned 70b fast-apply model with Fireworks, which built API support for Cursor's speculative edits, and reports about 1000 tokens per second.

Sourcegraph runs StarCoder for most Cody community-edition completions on Fireworks, so the team does not build its own inference stack.