Key Takeaways
- Fireworks.ai benchmarked Kimi K3, an open model, against Fable 5, a closed model, across about 1,030 agentic tasks.
- The post argues the two models are close on headline quality, but differ by task type, with K3 stronger on some terminal and symbolic work and Fable ahead in some coding-language breadth.
- Fireworks says routing tasks between the two models could produce better overall quality at a lower cost than using either model alone.
What happened
Fireworks.ai published a comparison of Kimi K3 and Fable 5 that frames the two models as close competitors rather than a clear winner-take-all story. According to the post, the team ran both models through the same harness on roughly 1,030 agentic tasks, with each benchmark aimed at a different kind of work.
The main claim is that Kimi K3 is a frontier-quality open model at a fraction of the cost, and that it complements Fable in a way that makes task routing attractive. Fireworks describes the idea of “oracle routing” as the theoretical best case: for each task, you pick the cheapest correct option after seeing which model would do best. In real deployments, however, a router has to predict the best choice in advance rather than trying both models.
On the headline numbers, the post says the models are very close. For SWE, the benchmark that Fireworks calls the headline one, K3 scored 92.4% and Fable 92.6%. Across five benchmark families, the post says the two models usually stay within a few points of each other, though Fable has a slight edge in multi-language breadth.
The more interesting difference appears when the results are broken down by task type. In SWE, Fireworks says K3 is strongest on symbolic math and developer tooling, while Fable does better on web and data visualization work. In the multi-language set, Fable is said to be stronger across Java, Python, and C++, while K3 is closer on JavaScript and Rust.
The post also highlights a class of long-horizon terminal tasks where K3 performed well. Fireworks says K3 completed tasks that Fable could not, including work involving a 7z hash, FEAL cryptanalysis, leaked secrets, a live vulnerability, and runaway async jobs.
The cost story is where Fireworks makes its strongest argument. It says the price gap between the two models is driven by token pricing, prompt caching, and effort per task. On SWE, K3 reportedly used far more effort than Fable, with about 55 turns and 1.3 million tokens per task versus 21 turns and 130,000 tokens for Fable. But prompt caching changes the economics, allowing K3 to remain cheaper even when it reads far more tokens.
For terminal-heavy tasks, the pattern can reverse. Fireworks says Fable can spiral into long runs, reaching 64 turns and 1.5 million tokens in some cases, sometimes timing out. The practical tradeoff is that extra turns can mean slower wall-clock times, even when the per-task cost looks better.
Why it matters
The post’s broader argument is that the best AI system may not be a single model chosen for every task, but a routing layer that directs work to the model best suited for it. If that routing is accurate enough, Fireworks says, the combined system can outperform either model alone while staying close to the cost of the cheaper option.
That matters because model selection is increasingly a product decision, not just a benchmark decision. A customer building agentic workflows may care less about which model wins by a fraction of a point on a single benchmark and more about how the model behaves across task types, how many turns it needs, and how quickly it finishes.
Fireworks’ conclusion is not that Kimi K3 replaces Fable or vice versa. Instead, it presents them as specialists with different strengths and different price-performance profiles. In that framing, the real opportunity is in orchestration: knowing when to use an open model, when to use a closed model, and when a router can make the decision automatically.
The post also suggests that task-level benchmarking may reveal more than aggregate scores. A near tie on a top-line benchmark can hide meaningful differences in the kinds of problems each model solves well. For teams building agent systems, those differences can matter as much as the average score.
What to watch
The biggest open question is whether the “oracle routing” insight can be turned into a practical router that works well on real workloads. Fireworks says its data suggests a near-perfect router may be possible, but it also notes that doing so would require far more routing data and real-world performance to prove it.
It will also be important to see how these results hold up outside this benchmark set. The source presents the findings as a snapshot from about 1,030 tasks, not a final answer about all model behavior. If future data shifts the balance on cost, latency, or task reliability, the routing strategy could change as well.
Another thing to watch is whether the speed tradeoff becomes decisive. The post argues that slower runs may be acceptable for background agents at scale, but not for use cases that need answers in a couple of seconds. That means the same cost advantage can look more or less attractive depending on product requirements.
For now, Fireworks’ report points to a broader shift in how frontier AI systems may be built: not around a single model that does everything, but around combinations of models optimized for different parts of the workload.
