A 2.78-Trillion-Parameter Model Now Runs in 8 GB of RAM and 179 KB of C. The Number That Matters Is 1.7 Terabytes

A single-developer project called kimi-k3-in-c runs inference on Moonshot AI's Kimi K3 — 2.78 trillion parameters, the largest open-weight model anyone has shipped — on one CPU, in 8.24 GB of peak resident memory, using a C99 engine that compiles to under 180 KB with no BLAS, no PyTorch, and no GPU code path. It has 8.1k stars and 1.3k forks, and it is Apache 2.0.

The claim sounds like it cannot be true. It is true, and the way it is true is more interesting than the headline.

The arithmetic

At bfloat16, K3's weights are about 5,560 GB. The repository walks the reduction down in four steps, each of which is a property of the model rather than a trick played on it.

The second step is the one doing the heavy lifting, and it is not the quantisation. K3 routes each token to 16 of 896 experts per layer. The other 880 are inert for that token. A conventional inference stack holds all of them in memory because moving weights is expensive; this project pays the I/O instead and keeps almost nothing resident. Moonshot's own architecture notes put active parameters at roughly 50 billion per token out of 2.8 trillion — which is to say the model was always 98% idle, and nobody had built an engine that took that literally.

Attention gets the same treatment. Of 93 layers, 69 use Kimi Delta Attention, whose recurrent state is a fixed 217 MB regardless of context length. The 24 global-attention layers use MLA, caching one 576-dimensional latent per position instead of 96 heads × 320 dimensions — the repo measures that as a 53× reduction. Neither is the author's invention. Both are K3 design choices whose memory consequences a GPU-first stack has no particular reason to exploit.

What is actually built here

The engineering that is the author's own is in the plumbing, and it is unusually careful. MXFP4 weights are multiplied in their packed form rather than dequantised first, which the repo calculates avoids 194 GB of memory traffic per token. A hand-written JSON scanner indexes 497,220 tensors across 96 safetensors shards in 0.27 seconds by reading headers only. The BPE tokenizer reimplements tiktoken to byte-identical output, verified by roundtrip across Unicode, emoji and code.

Most telling is the section listing five invariants that silently produce plausible-looking garbage if you get them wrong: A_log indexed per head rather than per channel, the sign of the UT-transform inverse, which of two KDA matrices keeps its diagonal, MLA's 64 rope dimensions that are cached but never rotated, and the router bias that steers expert selection while the unbiased sigmoid scores do the weighting. Nobody writes that list without having been burned by each item. The test suite gates generation behind three conformance checks against a PyTorch reference — teacher forcing, greedy decode, and incremental decode with cache — and the whole kernel and tokenizer layer is testable before you download a single weight.

What it costs

Thirty-two seconds per token is roughly two tokens per minute. A three-hundred-token answer is an afternoon. And 71% of runtime at the small end is storage reads, not arithmetic — the engine is waiting on disk, which is exactly what you would predict from a design that trades memory for I/O.

Two caveats on that table. The figures are measured on a two-socket AMD EPYC 7763 with 124 cores, where the presets cap the memory budget rather than change the machine. An actual 8 GB laptop has neither 124 cores nor that memory bandwidth, so the laptop row is a memory-budget result, not a laptop result. And the fidelity claims — byte-identical output across every preset — rest on the project's own test suite. That suite looks well-constructed, but no independent reproduction has been published.

Where the requirement actually went

Here is the line the headline elides: the project needs roughly 1.7 TB of free storage. That is 1.56 TB for the checkpoint plus a 109 GB packed trunk file you generate before first run.

The memory requirement did not disappear. It moved down one rung of the hierarchy, from DRAM to disk, and got slower by three orders of magnitude in the process. That is a legitimate and well-chosen trade — disk is cheap and DRAM is not — but it means "runs on 8 GB of RAM" describes the working set, not the barrier to entry. The barrier is a 1.7 TB download.

So the practical value is not that you will now serve K3 from a spare desktop. It is that the engine is a complete, dependency-free, readable statement of what K3 inference actually consists of, in the lineage of Karpathy's llama2.c: something you can read end to end and compile in seconds. For anyone porting K3 to unusual hardware, or trying to understand where a trillion-parameter MoE actually spends its bytes, a 179 KB C program with a five-item list of ways to get it subtly wrong is worth considerably more than a framework that hides all of it.

Sources: kimi-k3-in-c on GitHub · Kimi K3 model overview, Hugging Face · Moonshot AI unveils Kimi K3, CNBC