Kimi K3: Is It Really Open Source? And What Would It Take to Recreate It?
1. What Is Kimi K3?<br>Kimi K3 is the latest flagship model from Moonshot AI (Beijing), released July 16, 2026 . It is a Mixture-of-Experts model with 2.8 trillion total parameters and about 32 billion active per token (16 out of 896 experts). It supports 1 million tokens of context and handles text, images, and video natively.<br>On launch, K3 ranked #3 on the Artificial Analysis leaderboard behind Claude Fable 5 (Anthropic) and GPT 5.6 Sol (OpenAI), but #1 on Arena.ai for front-end coding .<br>2. Key Specs
Spec<br>Value
Total params<br>2.8T
Active params<br>~32B
Architecture<br>MoE
Experts<br>896 total, 16 active per token
Context<br>1M tokens
Multimodal<br>Text + Images + Video (native)
Attention<br>Kimi Delta Attention (KDA) + Attention Residuals (AttnRes)
MoE framework<br>Stable LatentMoE
Activation<br>Sigmoid Tanh Unit (SiTU)
Quantization<br>MXFP4 weights, MXFP8 activations (QAT from SFT onward)
Thinking<br>Always on (max effort by default)
Input price<br>$3.00/MTok (cache miss), $0.30/MTok (cache hit)
Output price<br>$15.00/MTok
3. Is It Actually Open Source?<br>Status as of July 17, 2026
Element<br>Public?<br>Details
Weights<br>Pending release by July 27, 2026<br>Announced by Moonshot
Technical report<br>Not yet published<br>"Coming soon"
API<br>Yes<br>OpenAI-compatible
Consumer apps<br>Yes<br>Kimi.com, Kimi Work, Kimi Code
Architecture code<br>No<br>KDA, AttnRes not released
Training code<br>No<br>No RL scripts
Training data<br>No<br>Neither pretraining nor post-training
The Pattern<br>Moonshot follows a consistent playbook:<br>K2 (July 2025) : Weights on HuggingFace with Modified MIT License. MAI training code released.<br>K2.5 (Jan 2026) : Same terms. Community accused them of not being truly open source.<br>K3 (July 2026) : Weights promised, but no training code or datasets.<br>Verdict : K3 is open-weight , not open source by OSI standards. You will be able to download and use the weights freely, but the entire training system stays proprietary. This is the same business model DeepSeek and other Chinese labs use: release weights for adoption, keep training details secret for competitive advantage.<br>4. Moonshot Timeline<br>Oct 2023: Kimi chatbot (128K context)<br>Mar 2024: Kimi 2M character context<br>Jan 2025: Kimi k1.5 (RL scaling, matched o1)<br>Apr 2025: Kimi-VL (16B MoE, open source)<br>Jun 2025: Kimi-Dev (72B coding), Kimi-Researcher<br>Jul 2025: Kimi K2 (1T params, 32B active, open-weight)<br>Sep 2025: K2-Instruct-0905 (256K context)<br>Oct 2025: Kimi Linear (48B, KDA preview)<br>Jan 2026: Kimi K2.5 (multimodal, Agent Swarm)<br>Apr 2026: Kimi K2.6 (1000+ parallel agents)<br>Jun 2026: Kimi K2.7 Code<br>Jul 16, 2026: Kimi K3 (2.8T params)<br>Jul 27, 2026: K3 weights release (promised)<br>5. What Would It Take to Recreate K3?<br>Assume you have the hardware (tens of thousands of H200/B200 GPUs with supernode interconnects of 64+ accelerators).<br>5.1 Architecture (Not Documented)<br>K3's architectural innovations are not publicly detailed :<br>Kimi Delta Attention (KDA) : A hybrid linear attention variant. Previewed in Kimi Linear (Oct 2025), but precise details are not public.<br>Attention Residuals (AttnRes) : Selectively recovering representations across network depth instead of accumulating them uniformly. Implementation unknown.<br>Stable LatentMoE : Routing framework with 896 experts (16 active). Includes Quantile Balancing that eliminates balancing hyperparameters.<br>Per-Head Muon : An extension of MuonClip that optimizes attention heads independently.<br>Sigmoid Tanh Unit (SiTU) : New activation function, not documented.<br>Gated MLA : Evolved version of Multi-Head Latent Attention.<br>Estimated effort : 6-12 months of reverse engineering just to reconstruct the architecture.<br>5.2 Pretraining Data<br>K3 was trained on a corpus of unknown size (estimated 20T+ tokens vs 15.5T for K2)<br>Proprietary rephrasing techniques<br>Training curriculum (LR scheduling, data proportions for multilingual/code/math, long-context phases) is entirely not public<br>5.3 Post-Training Data and Pipeline<br>Agentic data synthesis : Unknown scale, likely much larger than K2 (3000+ real tools, 20,000+ synthetic)<br>Preference data for initializing the critic model: internal only<br>Trajectory generation pipeline : not public<br>5.4 RL Algorithms and Hyperparameters<br>K3's technical report has not been published yet. Based on K2 and Kimi-Researcher, the unknowns include:
Parameter<br>Known?<br>Detail
RL algorithm<br>No<br>REINFORCE? PPO? GRPO?
Reward model<br>No<br>Not specified for K3
Gamma-decay factor<br>No<br>Unknown thresholds
Temperature decay schedule<br>No<br>Not published
Token budget limits<br>No<br>Not specified
Negative sample control ratio<br>No<br>Not published
Turn-level partial rollout<br>No<br>Precise mechanism unknown
Context management strategy<br>No<br>Needed for 1M tokens
5.5 Training Infrastructure<br>QAT with MXFP4/MXFP8 : proprietary implementation<br>Supernode config with 64+ accelerators : not documented<br>Fully balanced expert-parallel training : code not released<br>Asynchronous rollout system for agent RL: not public<br>Sandbox Kubernetes + MCP :...