Moonshot Publishes Kimi K3.1 Pro Open Weights, and vLLM Serves the 2.8T-Parameter Model on Day 0

Moonshot AI published the weights of Kimi K3.1 Pro on Hugging Face on Nov 4, 2026, and the vLLM team shipped day-0 serving support the same day. The model card describes an open-weight, natively multimodal agentic model — Moonshot’s most capable 3T-class model to date — post-trained from Kimi K3 with large-scale joint reinforcement learning across hundreds of downstream domains, and the first entry in the K3.1 series. Kimi’s account announced the weights at 4:00 PM that afternoon, and the moonshotai organization feed shows the repo published about three hours before our 18:49 UTC fetch.

That pairing is what makes this consequential rather than routine. Frontier-class open weights usually wait days for real serving support while engine teams decode a new architecture. Here the sparse-attention kernels, the packed cache layout, AMD, TPU and Alibaba PPU backends, and a reinforcement-learning training path all landed with the weights.

What the card documents

Spec	Value
Total / activated parameters	2.8T / 104B (MoE)
Layers	93: 69 KDA + 24 Gated DSA
Experts	896, 16 selected per token
Context window	1,048,576 tokens
Vision encoder	MoonViT-V2, 401M parameters
Quantization	MXFP4 weights, MXFP8 activations (quantization-aware training)
License	Kimi K3.1 Pro License

One architectural change from Kimi K3 carries the serving story: the 24 full-attention layers replace dense Gated MLA with Gated DeepSeek Sparse Attention for more efficient long-context modeling.

On benchmarks, the card reports 65.7 on Terminal-Bench 4.0 with K3.1 Pro running in the Claude Code harness, against 66.4 for Claude Opus 5.5, 58.2 for GPT-6 Astra and 57.9 for Claude Fable 5.1 — plus 51.7 on SWE Marathon v1.1, the best score in its comparison set, and 94.2 on GPQA-Diamond. Read these as vendor-reported: several benchmarks are Moonshot internal, competitor numbers are max-effort values drawn from other labs’ posts, and the card’s footnotes spell out how much harness choice matters in agentic evals.

The license deserves a flag. This is the custom Kimi K3.1 Pro License, not an Apache-style open-source license; anyone planning to serve or fine-tune should read its terms before building on the weights.

What day-0 serving buys

The vLLM engineering post is concrete about why a 2.8T MoE is servable at 1M context. Each sparse layer carries a small indexer: one FP8 key per token (132 bytes with its scale), scored by 32 lightweight heads. It keeps the top 2,048 tokens, and MLA then runs exactly — over only those tokens. At long context a decode step reads about 23 percent of the bytes per layer that dense MLA reads. That is the difference between 1M-context decode being bandwidth-prohibitive and being a capacity-planning problem.

The index keys form a second per-token cache, which vLLM packs into one block-row layout together with the MLA latent and KDA state. FP8 latent plus FP8 index keys costs 16,992 bytes per token per GPU across the 24 sparse layers, against 13,824 for Kimi K3; an NVFP4 latent with MXFP4 index keys drops it to 10,080. On eight GB300 GPUs that format holds 1.7 times more tokens than FP8, and decode context parallelism multiplies capacity by seven — roughly 36 full 1M-token contexts on one node, against 3 under TP8 with FP8.

Serving result (8x GB300 unless noted)	Number
Prefill TTFT at 340K tokens, token-sharded vs head-sharded	9.42 s vs 15.89 s (1.69x)
Time per output token, concurrency 1, 32K context	9.35 ms TP8; 5.96 ms DCP8 with fused decode
KV capacity, TP8 vs DCP8 (FP8)	3.04M vs 21.8M tokens
AgentX on GB300: latency / throughput ends	375.9 tok/s per user; 37.8K tok/s per GPU
AgentX on Rubin: latency / throughput ends	815.6 tok/s per user; 68.7K tok/s per GPU

Day-0 coverage is unusually broad: NVIDIA GB300 and Rubin, AMD Instinct MI355X, a 16-chip TPU v7x slice and — a first — Alibaba’s T-Head Zhenwu PPU via the vllm-sail plugin. The quick start is one command:

vllm serve moonshotai/Kimi-K3.1-Pro \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --load-format fastsafetensors \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

RL shipped on day 0 too

For the first time, a vLLM day-0 post includes a reinforcement-learning path: LoRA RL with verl, vLLM as the rollout engine and Megatron as the trainer, producing bit-identical log-probs for the base model and a rank-32 adapter spanning every attention projection and all 896 experts. Over a 100-step full-model run the rollout/trainer KL stayed exactly 0 while AIME mean@8 rose from 0.451 to 0.664. The usual train/serve mismatch debugging is eliminated at the kernel level, not bounded by importance sampling. A DSpark speculative-decoding preview checkpoint, Inferact/Kimi-K3.1-Pro-DSpark-preview, adds a claimed 3.0x decode speedup at batch size 1 — explicitly a preview, not a final drafter.

What operators should do

• Want the model without the hardware? It is live as kimi-k3.1-pro on platform.kimi.ai with OpenAI- and Anthropic-compatible endpoints; reasoning_effort accepts low, high and max (default max).
• Mind the multi-turn contract. The card requires passing the complete assistant message back — reasoning_content and tool_calls included, not just content. Harnesses that rebuild messages will degrade the model’s thinking.
• Reality-check self-hosting. Every number above comes from GB300, Rubin, MI355X, TPU v7x or Zhenwu PPU-class hardware. The serve command is one line; the cluster behind it is not.
• If you fine-tune, test the zero-mismatch RL recipe and the DSpark preview before committing a training run.
• Read the license first. It governs serving and fine-tuning, not just downloads.

K3.1 Pro lands in an open-weight field that already includes Kimi K3, GLM and DeepSeek V4 — see our open-weight frontier scorecard — but it is the first of that class to arrive with its serving stack on day 0. Whether self-hosting beats metered API spend still comes down to pricing per completed task, not per token, and Kimi’s agent-side record is covered in our guide to Chinese models for agent work.

Load X post

Loading this post connects to X.

The original source is available above.