AI teams already train, fine-tune and serve their models on Verda, from dedicated GPU clusters to serverless containers that scale with demand. Today, we’re adding the layer above. Verda Inference is our model-as-a-service (MaaS) offering: managed inference for frontier open-weight models, built for AI teams who need consistent performance and reliability for their applications and agents at scale. We handle the GPUs, the models and the serving in our own data centers in Europe, so you can focus on what you’re building and pay only for the tokens you use.
We’re launching in partnership with Moonshot AI, starting with Kimi K3.1 Pro, the highest-ranked open-weight model on the Artificial Analysis Intelligence Index. As a European launch partner of Moonshot AI, we’ll bring new Kimi models and versions to Verda Inference as their open weights are released, for builders in Europe and around the world. For developers building on open-weight models, that means access to Moonshot’s newest releases from day zero, so you can always build with the most capable models and stay at the frontier.
Kimi K3.1 Pro is built for long, agentic work, and serving that work well is hard. One slow or failed request can derail an agent’s entire task, and the longer a session runs, the more context it carries. We built Verda Inference for this, with performance engineered for every request, from a single prompt to agents that run for hours. On long-context agentic workloads, it delivers up to 10x faster time to first token than standard caching.* Your agents can return to a million-token session and pick up their work in seconds, not half a minute, so long tasks finish sooner.
*Based on internal testing with Kimi K3.1 Pro, using a coding agent context of one million tokens, comparing offloaded and recomputed context on the same deployment.
Serving everything from quick fixes to full build
Kimi K3.1 Pro is the first model in Moonshot AI’s new K3.1 series, and its most capable 3T-class open-weight model to date, with a 1 million token context window. It brings major improvements in coding, agentic work, 3D creation and financial research, and it’s strongest at long-horizon coding, where agents plan, build, test and revise over many steps. In Moonshot’s own benchmarks, it matches or approaches the strongest proprietary models across coding, agentic and knowledge tasks.
For inference, a quick fix and a full build are very different jobs. One needs a fast first response. The other can run for hours, resending a growing context at every step, and needs performance that holds as the session grows and requests that don’t fail partway through. Verda Inference is built to serve both well, on the same model and at the same time.
Performance engineered for every request
Agents resend their growing context with every step, often 100,000 tokens or more per request, and most of it repeats from the step before. Serving them well takes more than fast GPUs. The two hardest parts are reusing that context instead of processing it again, and keeping one agent’s long prompt from slowing down everyone else.
Verda Inference reads context once and reuses it. Each session’s context stays in a shared cache, and when your agent returns for its next step, only the new part is processed. In our testing, the cache reused about 99% of the context agents sent again. Your agent gets its first token sooner, and because cached input is priced lower, long sessions cost less.
When a long session’s context no longer fits in GPU memory, we offload it to a larger memory tier and load it back when the agent returns, instead of recomputing it from scratch.
We also keep long prompts from blocking answers. Processing a long-context prompt is heavy work, and if the same GPUs are also writing answers, everyone waits. We read prompts and write answers on separate GPUs, and send each answer to the least busy server, so output keeps flowing even when long prompts arrive.
Running our own GPUs, with serving software we build and tune for them, lets us fix a bottleneck anywhere in the stack. In six weeks, our engineering team fit 16 times more agent sessions on each GPU, on the same hardware.
*Measured on SemiAnalysis’s AgentX agent workload. Prompts averaging 173,000 tokens. Cache share as reported by the serving engine.
A model is only as good as the way it’s served
Before a model reaches customers, we test it under sustained load, with many long-context agentic sessions running at once, against strict targets for response time, output speed and failure rate. Reusing saved work doesn’t change the answers: in our testing, responses served from offloaded context matched freshly computed ones exactly.
In production, we protect the sessions already running. When a model reaches capacity, it stops taking on new projects, so the projects already using it keep running. And because we run the data centers as well as the GPUs and software, our team monitors the whole stack from the hardware up, and can fix problems quickly.
Run everything from coding agents to everyday features
- Coding agents: agents that read large codebases, make changes, run tests and keep revising until the work is done. Such sessions can last for hours, resending the whole history at every step, and our caching keeps them fast and affordable.
- Research and analysis: analyze long reports, contracts and datasets in a single request with long-context models, so the model works across everything at once instead of one piece at a time.
- Agent workflows: plan with the most capable model, then hand routine steps to faster, lower cost ones as they join the platform. Everything runs through the same API, so each request can use the best model for the step.
- Product features: power chat, summaries and classification inside your product, with each feature running on the model that gives it the right balance of quality, speed and cost.
Kimi K3.1 Pro in action on Verda Inference
We put Kimi K3.1 Pro to work on our own code. Here’s one session, step by step, in which a coding agent running on the model checked a design document for one of our services against the code. Over 11 steps, the agent explored the codebase, read the document and verified every claim in it.
As the agent’s context grew past 25,000 tokens, first tokens kept arriving in about 150 milliseconds on average. 92% of its input was served from cache. The rest was new material, such as files it had just read, which always has to be processed fresh. Of everything the agent had already sent, our cache reused about 99%.*
Recorded coding agent session with 11 model calls, replayed at an accelerated pace. Context grows from 9.9K to 27.3K tokens. 92% of input tokens are served from cache. Median time to first token is 148 milliseconds; the maximum is 161 milliseconds.
| Call | Tool | Action | Context tokens | Cached input | Time to first token |
|---|---|---|---|---|---|
| 01 | search | List repo, find router and tier code | 9,925 | 98% | 144 milliseconds |
| 02 | read | List global-router, read README | 11,675 | 85% | 148 milliseconds |
| 03 | search | Grep tier in gateway upstream | 14,731 | 80% | 149 milliseconds |
| 04 | read | Read admission.rs, list tmp/, design/ | 16,417 | 90% | 148 milliseconds |
| 05 | read | Read existing per-tier doc | 18,032 | 91% | 161 milliseconds |
| 06 | read | Read doc lines 100–280 | 20,470 | 88% | 147 milliseconds |
| 07 | read | Read doc lines 280–415 | 22,666 | 90% | 54 milliseconds |
| 08 | search | Verify queue keys and defaults | 24,545 | 92% | 57 milliseconds |
| 09 | read | Verify transfer(), gateway policy | 25,930 | 95% | 159 milliseconds |
| 10 | search | Verify headers and mutex layout | 26,707 | 97% | 158 milliseconds |
| 11 | reply | Report: doc verified, no changes | 27,308 | 98% | 150 milliseconds |
Kimi K3.1 Pro can take on real engineering work, like exploring a codebase and checking a document against it claim by claim, without step by step guidance. Verda Inference keeps that work practical as it grows: the agent was as quick to respond at step 11 as at step 1, and because repeated context is served from cache at a fraction of the price, a long session costs far less than it would if every step were processed from scratch.
*One recorded session on Verda Inference
Built for AI builders
- Compatibility with your existing code: With an OpenAI compatible API, moving your code over means changing the base URL, API key and model ID. Your agents, frameworks and tools keep working as they are.
- Choice of frontier open-weight models: Verda Inference is built to serve a range of leading open-weight models through one API. You can use the right model for each task and switch between them by changing one line of code.
- Pricing that rewards long sessions: You pay only for the tokens you use, deducted from your prepaid Verda balance like the rest of the platform.
- One console for inference and compute: You create API keys and track token usage, including cached tokens, in the same Verda console as your instances, clusters and storage.
Get started in three steps
- Enable Kimi K3.1 Pro: In the Verda console, open Inference, choose Kimi K3.1 Pro, and enable it. Your API key is created in the same step.
- Point your code or agent at Verda Inference: Set the base URL to
https://inference.verda.com/v1, and use your new API key and the model IDkimi-k3-1-pro. Coding agents such as OpenCode and Pi Coder have ready-made setups in the console. - Make your first call: Send a request, or start your agent on a task.
You can also try Kimi K3.1 Pro in the Playground first, before connecting your code.
Try Verda Inference in preview
Verda Inference is available in preview from today, starting with Kimi K3.1 Pro. See the pricing page for current prices.
We’ll make more leading open-weight models available soon, along with service tiers for everything you need, from faster responses to lower cost batch work. Tell us what you’re building, and what you’d like us to add next.
Sign in to the Verda console to make your first call, or read the documentation to learn more. Usage limits apply during the preview. Planning a larger workload? Get in touch with our team to help you size it and get it running.
