Run Kimi K3 and K3.1 Pro Locally: Hardware and K3 Speed Tests

Kimi K3.1 Pro weights went public on Hugging Face on November 2, and they need exactly as much disk as Kimi K3. Both ship as 1.56 TB of MXFP4 safetensors in 96 shards. What changed is what can read them. You can run Kimi K3 locally today from a 594 GB 1-bit GGUF in llama.cpp, slowly, but K3.1 Pro has no GGUF, no llama.cpp support and no released vLLM wheel yet.

Original content from computingforgeeks.com - post 170430

This guide keeps our K3 run on four A100 GPUs with 2 TB of RAM, where generation crawled at 0.10 tokens per second. It adds what you can check about K3.1 Pro before renting anything: the shipped checkpoint, memory for weights plus KV cache, which engines load it, and the license. Every K3.1 check below was run on an Ubuntu 24.04 VM in November 2026, so the engine status is dated rather than guessed.

Kimi K3 and K3.1 Pro in numbers

K3.1 Pro is a post-trained K3 with one structural change, and that change is the whole reason the local tooling lags. The K3.1 Pro model card says Moonshot replaced K3’s full-attention Gated MLA layers with Gated DeepSeek Sparse Attention (Gated DSA) layers. Everything else, the expert count, the active parameters and the context window, carries over:

Property	Kimi K3	Kimi K3.1 Pro
Total parameters	2.8T (llama.cpp reads 2779.48 B from the GGUF)	2.8T
Active per token	104B, 16 experts selected from 896	104B, 16 of 896 plus 2 shared
Layers	93	93: 69 KDA and 24 Gated DSA
Attention	Kimi Delta Attention plus Gated MLA	Kimi Delta Attention plus Gated DSA
Context window	1,048,576 tokens	1,048,576 tokens
Vision encoder	MoonViT-V2, 401M parameters	MoonViT-V2, 401M parameters
Official checkpoint	MXFP4, 96 shards, 1.56 TB	MXFP4, 96 shards, 1.56 TB
GGUF quants	Unsloth, 594 GB to 1,561 GB	None as of November 5
License	Kimi K3 License	Kimi K3.1-Pro License

The mixture-of-experts design is what makes either model theoretically approachable. Only 104B parameters activate per token even though 2.8T sit in memory. That helps compute. It does nothing for the memory bill, because the router can pick any expert at any token, so all of them have to stay resident. llama.cpp labels the K3 GGUF 2.8T.A50B, which is its own accounting of the active weight, so do not be thrown when the loader disagrees with the model card.

Check what Moonshot shipped for K3.1 Pro

You do not need to pull 1.56 TB to learn what you are dealing with. The config, index and license files are a few hundred kilobytes, and they answer the questions that decide whether a box can host the model. On a fresh Ubuntu 24.04 machine, install jq, git and the Hugging Face CLI first. The CLI goes into a virtualenv because Ubuntu marks its system Python as externally managed:

sudo apt install -y python3-venv jq git
python3 -m venv "${HOME}/.venvs/hf"
"${HOME}/.venvs/hf/bin/pip" install -U huggingface_hub
export PATH="${HOME}/.venvs/hf/bin:${PATH}"

Then fetch only the small files:

hf download moonshotai/Kimi-K3.1-Pro --include "*.json" --include LICENSE --local-dir ~/k31

Each pattern needs its own --include flag, which the next subsection explains. With the files on disk, jq pulls out the architecture name and the layer split:

jq -c '{model_type, architectures}' ~/k31/config.json
jq '.text_config | {num_hidden_layers, num_experts, num_experts_per_token, kv_lora_rank, max_position_embeddings, dsa_layers: (.dsa_config.dsa_layers | length), kda_layers: (.linear_attn_config.kda_layers | length)}' ~/k31/config.json

Read the model type first, because kimi_k31 and KimiK31ForConditionalGeneration are the strings an engine has to recognise before it loads anything:

Those 24 DSA layers sit at every fourth position plus the last one, and the remaining 69 are KDA linear attention. The KDA layers keep a fixed-size state per sequence, while the DSA layers keep a compressed latent per token, sized by kv_lora_rank. That split is what the memory math in the next section depends on.

Warning: “Ignoring `–include` since filenames have been explicitly set”

Our first attempt passed both patterns after one flag, --include "*.json" LICENSE, and the CLI quietly treated LICENSE as an explicit filename. It then threw the include pattern away and fetched one file:

The command still exits cleanly, so a script will not notice. On a small file set that costs nothing, but the same mistake on a weights pull (--include "*.safetensors" config.json) downloads the config and skips 1.56 TB of weights without an error. Repeat --include for every pattern, as in the command above.

Confirm the checkpoint format and size

Moonshot shipped one checkpoint, quantised with compressed-tensors. Read the quantisation block straight from the config:

jq -c '.text_config.quantization_config | {quant_method, format, num_bits: .config_groups.group_0.weights.num_bits, group_size: .config_groups.group_0.weights.group_size}' ~/k31/config.json

Hugging Face’s size badge rounds, and the repository also carries tokenizer and image-processor files, so we summed only the safetensors shards through the Hub API. Create a small script for it:

vim k31_size.py

Paste the following. It takes a repo ID as its only argument and defaults to K3.1 Pro:

import sys
from huggingface_hub import HfApi

repo = sys.argv[1] if len(sys.argv) > 1 else "moonshotai/Kimi-K3.1-Pro"
info = HfApi().model_info(repo, files_metadata=True)
shards = [s for s in info.siblings if s.rfilename.endswith(".safetensors")]
total = sum(s.size for s in shards)

print(f"repo:     {repo}")
print(f"created:  {info.created_at:%Y-%m-%d}")
print(f"gated:    {info.gated}")
print(f"shards:   {len(shards)}")
print(f"bytes:    {total:,}")
print(f"size:     {total / 1e12:.2f} TB ({total / 2**40:.2f} TiB)")
if info.safetensors:
    for dtype, count in info.safetensors.parameters.items():
        print(f"params {dtype}: {count / 1e12:.2f}T")

Run it for K3.1 Pro, then for K3 so you can compare the two releases side by side:

python k31_size.py
python k31_size.py moonshotai/Kimi-K3

Both repos come back ungated with 96 shards and 1.56 TB (1.42 TiB). Hugging Face counts 2.72T parameters as packed U8 MXFP4 storage and 0.06T as BF16, which is the attention, shared experts, router and vision tower that the quantisation config leaves unquantised:

There is no BF16 or FP8 build of K3.1 Pro to fall back on, and the model card gives no sign of one, because it lists the MXFP4 weights as the product of quantisation-aware training. The 4-bit checkpoint is the native model. That also means a future GGUF can only get smaller by going below 4 bits, which is exactly what Unsloth’s 1-bit and 2-bit K3 quants did.

Size the memory for K3.1 Pro

K3.1 Pro needs about 1.6 TB of GPU memory for a single long conversation and more than 1.8 TB once you serve several full-length ones. The weights set the floor at 1,561 GB, and the KV cache adds less than you might expect, because only 24 of the 93 layers keep a per-token cache. We wrote the arithmetic as a script that reads the same config.json, so it stays correct if Moonshot updates the file:

vim k31_mem.py

The two assumptions are written into the comments: each DSA layer caches a BF16 MLA latent plus a BF16 indexer key per token, which is an upper bound, and each KDA layer holds an FP32 recurrent state per sequence:

import json
import sys

root = sys.argv[1] if len(sys.argv) > 1 else "."
cfg = json.load(open(f"{root}/config.json"))["text_config"]
weights = json.load(open(f"{root}/model.safetensors.index.json"))["metadata"]["total_size"]

dsa = len(cfg["dsa_config"]["dsa_layers"])
kda = len(cfg["linear_attn_config"]["kda_layers"])
heads = cfg["linear_attn_config"]["num_heads"]
hdim = cfg["linear_attn_config"]["head_dim"]

# BF16 MLA latent plus BF16 indexer key per DSA layer per token (upper bound)
per_token = dsa * (cfg["kv_lora_rank"] + cfg["qk_rope_head_dim"]) * 2
per_token += dsa * cfg["dsa_config"]["index_head_dim"] * 2
# FP32 recurrent state per KDA layer, fixed per sequence
kda_state = kda * heads * hdim * hdim * 4

GB = 1e9
print(f"weights on disk : {weights / GB:,.1f} GB")
print(f"DSA layers {dsa}, KDA layers {kda}")
print(f"KV per token    : {per_token:,} bytes")
print(f"KDA state/seq   : {kda_state / GB:.2f} GB")
print(f"{'context':>10} {'cache/seq GB':>13} {'1 seq total GB':>15} {'8 seqs total GB':>16}")
for ctx in (32_768, 131_072, 262_144, 1_048_576):
    cache = ctx * per_token + kda_state
    print(f"{ctx:>10,} {cache / GB:>13.1f} {(weights + cache) / GB:>15,.0f} {(weights + 8 * cache) / GB:>16,.0f}")

Point it at the directory holding the downloaded config and index:

python k31_mem.py ~/k31

Our script puts the cache at 33,792 bytes per token plus 0.43 GB of KDA state per sequence. A single 1M-token conversation therefore adds about 36 GB on top of the weights:

Eight concurrent sequences at the full window come to 1,848 GB, close to the 1,874 GB minimum in vLLM’s K3.1 recipe, which its YAML derives as the 1,561 GB of safetensors times 1.2. Engines also need room for activations and CUDA graphs, so treat the script’s totals as a floor rather than a budget. On real eight-GPU nodes, those numbers land like this:

• 8x H200 at 141 GB each: 1,128 GB, short by more than 400 GB
• DGX B200 at 1,440 GB total: short of the weights alone
• One B300 or MI355X node, or two GB300 trays of four GPUs, at 288 GB per GPU: 2,304 GB, the NVIDIA and AMD baselines in vLLM’s recipe

Anything smaller has to offload experts to system RAM, which is how we ran K3 on four A100s. None of the K3.1 recipes documents that, and we did not test it, so for now the realistic local K3.1 box is eight GPUs of the newest generation. vLLM’s recipe also lists Rubin VR200 and two Alibaba accelerator configurations, none of which you can buy for a home lab. The script’s per-token figure drops further if the engine stores the cache in FP8 or NVFP4, and vLLM’s recipe offers an nvfp4_ds_mla KV cache option for exactly that reason.

Check which engines load K3.1 Pro

As of November 5, SGLang’s main branch is the only engine with K3.1 Pro model code merged, vLLM ships it only as dedicated Docker images while its model PR is still open, and llama.cpp has nothing. Engine support changes daily in a release week, so these are the checks we ran and you can repeat. Start with GGUF, since that is what llama.cpp, Ollama and LM Studio need:

hf models ls --search Kimi-K3.1 --filter gguf
hf models ls --search Kimi-K3.1 --sort downloads --limit 8 --format json | jq -r '.[].id'

The GGUF filter returns nothing. The unfiltered search shows the official repo, draft models for speculative decoding and a handful of re-uploads and fine-tunes, all in safetensors, so they need the same engines the original does:

A missing GGUF usually means the converter cannot handle the architecture yet, so check the engines’ source next. We shallow-cloned all three projects and grepped each for the K3.1 class name:

• ggml-org/llama.cpp, runtime and converter
• vllm-project/vllm, model registry and recipes
• sgl-project/sglang, sparse clone of the models directory

Clone them into your home directory. The SGLang checkout stays sparse because its full tree is large and only the models directory matters here:

cd ~
git clone --depth 1 https://github.com/ggml-org/llama.cpp
git clone --depth 1 https://github.com/vllm-project/vllm
git clone --depth 1 --filter=blob:none --sparse https://github.com/sgl-project/sglang
cd ~/sglang && git sparse-checkout set python/sglang/srt/models

Then search each tree:

grep -n '"kimi-' ~/llama.cpp/src/llama-arch.cpp
grep -rlE 'kimi.?k31|KimiK31' ~/llama.cpp/src ~/llama.cpp/conversion | wc -l
grep -c KimiK31 ~/vllm/vllm/model_executor/models/registry.py
grep -n KimiK31 ~/sglang/python/sglang/srt/models/kimi_k3.py

llama.cpp master at commit 330f018 knows kimi-k3 but has no K3.1 code in the runtime or the converter. vLLM main has no KimiK31 entry in its model registry. SGLang main defines KimiK31ForConditionalGeneration as a subclass of its K3 model:

Source on a main branch is not the same as something you can pip install, so the last check compares the open vLLM PR with what PyPI actually ships:

curl -s https://api.github.com/repos/vllm-project/vllm/pulls/65685 | jq -r '"#\(.number) \(.state) merged=\(.merged): \(.title)"'
curl -s https://pypi.org/pypi/vllm/json | jq -r '.info.version'
curl -s https://pypi.org/pypi/sglang/json | jq -r '.info.version'
grep -nE 'min_vllm_version|nightly_required|pip:|vram_minimum_gb|nvidia:' ~/vllm/recipes/models/moonshotai/Kimi-K3.1-Pro.yaml

The model PR is open, PyPI carries vLLM 0.33.0, and the merged recipe asks for 0.34.0 with pip: false on every hardware target:

Put together, this is where each engine stands for K3.1 Pro on November 5, 2026:

Engine	Kimi K3	Kimi K3.1 Pro
llama.cpp	kimi-k3 in master; our numbers use the Unsloth fork	No code in master, no PR found, no GGUF on Hugging Face
vLLM	KimiK3ForConditionalGeneration in the registry	PR #65685 open; Docker images vllm/vllm-openai:kimi-k31 (CUDA 13, pushed November 4) and vllm/vllm-openai-rocm:kimi-k31; no wheel
SGLang	kimi_k3.py model	Basic support merged to main in PR #48415 on November 4; PyPI 0.5.23 is from October 30 and predates it
Ollama, LM Studio	Not tested by us	Nothing to load until a K3.1 GGUF exists

Moonshot’s card also lists TokenSpeed as a serving option, which we did not check. If you have the hardware today, the vLLM Docker image or an SGLang build from main are the two routes with merged recipes behind them. The vLLM K3.1 Pro recipe carries the flags, including the kimi_k3 tool-call and reasoning parsers that K3.1 reuses. We did not run either, since none of our hardware comes close to the 1.6 TB floor.

Work out the memory you need for Kimi K3

Kimi K3 is the model you can actually run on less than a full B300 node, because Unsloth publishes dynamic GGUF quants of it. The sizes are the real constraint. We summed the file sizes for every shard through the Hugging Face API rather than trusting the summary table:

Quant	Size on disk	RAM plus VRAM needed
UD-IQ1_S	594.0 GB (14 shards)	610 GB
UD-IQ1_M	648.9 GB	665 GB
UD-IQ2_XXS	711.1 GB	726 GB
UD-Q2_K_XL	861.3 GB	880 GB
UD-Q4_K_XL	1,508.7 GB	1.56 TB
UD-Q8_K_XL	1,561.2 GB	1.6 TB

The largest quant is the same 1.56 TB as Moonshot’s own MXFP4 release, which is a useful sanity check: the native weights are already 4-bit, so a “Q8” file cannot hold more information than the original. Anything above UD-Q2_K_XL buys you very little over running the official checkpoint in vLLM, if you have the GPUs for that.

Combined memory is what counts, not VRAM alone. Given the flag covered in the serving section below, llama.cpp keeps the expert weights in system RAM and runs those tensors on the CPU backend, which is the only reason a four-GPU box with 160 GB of VRAM can load a 594 GB model at all. Nothing is streamed across the bus; the experts simply execute where they sit. Whether the result is usable is a separate question, and the benchmark section answers it.

Our test box was four A100-SXM4-40GB cards, two AMD EPYC 7542 processors giving 128 threads, and 2015 GiB of RAM. If you’re sizing your own machine, the difference between VRAM and pooled system memory matters more here than in any other local model, and the unified memory versus VRAM breakdown covers why. For single-card budgets and normal model sizes, the VRAM-to-model-size map and the GPU rankings for local LLMs cover the sane end of the market.

Download the Kimi K3 weights

Every command here runs as root, which is what rented GPU images hand you. On a machine where you’re not root, prefix the package install with sudo and point K3_DIR at a path your user owns. Set the paths once so the rest of the commands stay short:

export K3_DIR=/root/k3
export K3_QUANT=UD-IQ1_S
export K3_MODEL="${K3_DIR}/${K3_QUANT}/Kimi-K3-${K3_QUANT}-00001-of-00014.gguf"

The Hugging Face CLI is the fastest way to move this much data. Transfers now run over the Xet backend, which ships as a base dependency. Plenty of guides still tell you to install the hf_transfer extra and export HF_HUB_ENABLE_HF_TRANSFER; that extra no longer exists in current huggingface_hub releases and the variable is a deprecated no-op. Ubuntu marks its system Python as externally managed, so install into a virtualenv rather than fighting pip:

apt install -y python3-venv
python3 -m venv "${HOME}/.venvs/hf"
"${HOME}/.venvs/hf/bin/pip" install -U huggingface_hub
export PATH="${HOME}/.venvs/hf/bin:${PATH}"
export HF_XET_HIGH_PERFORMANCE=1
hf download unsloth/Kimi-K3-GGUF --include "${K3_QUANT}/*" --local-dir "${K3_DIR}"

Most rented GPU images already put you inside a conda or virtualenv Python, in which case the first three lines are unnecessary. On a datacenter link the full 594 GB landed in 235 seconds, a sustained 2.5 GB/s. Budget far more on a home connection. At full line rate the same transfer needs about 80 minutes over 1 Gbps and past 13 hours over a typical 100 Mbps residential link, and since real throughput lands nearer 85 to 90 percent of that, plan for roughly 90 minutes and 15 hours respectively. Confirm the shard count before moving on:

du -sh "${K3_DIR}/${K3_QUANT}" && ls "${K3_DIR}/${K3_QUANT}" | wc -l

Fourteen shards totalling 554 GiB is a complete UD-IQ1_S set:

554G	/root/k3/UD-IQ1_S
14

Keep the shards where they are. llama.cpp reads the whole set from the first file, so there’s no merge step to run.

Build llama.cpp with the Kimi K3 support

Mainline llama.cpp couldn’t read this architecture when we ran these tests, and Unsloth’s branch was the one that worked. Master has since gained a kimi-k3 architecture (it shows in the source check above). We have not re-run the numbers on it, so the steps below stay on the exact fork commit that produced them. The branch also carries the MoonViT vision tower, which this guide doesn’t cover, because the download above pulls only the quant directory and skips the separate projector file that vision needs. Clone the fork into its own directory, so it never collides with a master checkout, and check out the pull request branch:

git clone https://github.com/unslothai/llama.cpp llama.cpp-unsloth
cd llama.cpp-unsloth
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision
git checkout daef2b3e1b5b1ac7b575f13b13a9450cb2d02862

That last line is optional but recommended. The pull request head still moves, and daef2b3e is the exact commit, build b10245, that produced every number below.

Build with CUDA on. Setting the architecture to match your cards cuts the compile time by a large margin, and 80 is correct for A100. Use 89 for Ada and 90 for Hopper. Blackwell splits in two: 100 covers the B100, B200 and GB200 datacenter parts, while 120 covers the RTX 50 series and the RTX PRO 6000. Both Blackwell targets need CUDA 12.8 or newer. A bare architecture number compiles PTX alongside the device code, so a card newer than your target still runs the binary by jitting it, but the reverse doesn’t hold: build for 90 and run on an A100 and you get no kernel image available for execution on the device.

cmake -B build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=80
cmake --build build --config Release -j $(nproc) \
  --target llama-cli llama-server llama-bench llama-gguf-split

That took 248 seconds across 128 cores. Run the download and the build at the same time, because neither one competes for the other’s resources and together they’re the bulk of your setup cost. If CUDA isn’t working yet on the host, our NVIDIA driver and CUDA install guide gets that sorted first, and the llama.cpp cheat sheet lists the server and bench flags used below.

Load and serve the model

This is the configuration that actually loaded on a 160 GB VRAM box. The MoE weights go to the CPU, everything else goes to the GPUs:

./build/bin/llama-server \
  --model "${K3_MODEL}" \
  --n-gpu-layers 99 --cpu-moe --no-mmap \
  --ctx-size 8192 --temp 1.0 --top-p 0.95 \
  -t $(nproc) --host 127.0.0.1 --port 8080

Two flags carry the weight here. --cpu-moe keeps every layer’s expert tensors in system RAM, which is what lets the model fit. --no-mmap reads the whole file into memory instead of paging it from disk on demand, and with 2 TB of RAM available that’s the right call because paging 554 GiB through the page cache on every token is ruinous. The load took about nine minutes.

Two warnings about that second flag. llama.cpp now prints a deprecation notice steering you to --load-mode, and the true equivalent of --no-mmap is --load-mode none, not the --load-mode mmap the message appears to suggest. More important: drop --no-mmap entirely if your combined RAM and VRAM is smaller than the quant. Memory mapping is exactly what lets an undersized machine fall back to reading weights off disk instead of being killed by the OOM reaper. It’ll be painfully slow, but it will run.

The sampling values are the ones Moonshot recommends. Temperature 1.0 with top-p 0.95 for single-step work, top-p 1.0 for agentic loops. Confirm the server actually answers before spending real time on it:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Say hello."}],"max_tokens":16}'

Two things to expect from that call. At the measured rate those sixteen tokens take roughly two and a half minutes to come back, which is the first honest signal of what the rest of this will feel like. And no answer comes back. K3 is a thinking-only model, so a cap that low is spent entirely on the thought. Depending on whether the fork’s parser recognises the K3 thinking tags, you get an empty content with the thought extracted into reasoning_content, or a content full of raw thinking. Either way there’s no reply at sixteen tokens, and the server is working. That behaviour becomes a real problem later, and the quality section returns to it.

What Kimi K3 actually runs at

Prompt processing came in at 12.87 tokens per second under llama-bench, and between 5.5 and 6.2 tokens per second through the server on short prompts. Token generation is where the story ends: 0.10 tokens per second, held steady across the sampled window.

One token every ten seconds isn’t a usable chat model. A 500 token answer takes an hour and twenty three minutes. We aborted the llama-bench generation test after 17 minutes because it hadn’t finished 128 tokens, and a single short question never produced a complete answer inside the rental budget.

The cause is memory bandwidth, not compute. GPU utilisation sat between 0 and 1 percent for the whole run with only 64 GB of the 160 GB VRAM in use. Every token has to pull its active expert set out of system RAM, and that read is the wall. Adding GPUs doesn’t help unless you add enough of them to hold the entire model.

Two tuning attempts changed nothing worth reporting. Passing --numa distribute on this eight-node NUMA machine produced 5.85 to 6.12 tokens per second on prompt processing against 5.54 to 6.23 without it, which is inside the noise. The kernel also flagged numa_balancing as enabled and performance-impairing, and that file is read-only inside a container, so it couldn’t be turned off.

Error: “cudaMalloc failed: out of memory” allocating 96937.09 MiB on device 3

Trying to keep the last 15 layers’ experts on the GPUs with --n-cpu-moe 78 fails immediately:

ggml_backend_cuda_buffer_type_alloc_buffer: allocating 96937.09 MiB on device 3: cudaMalloc failed: out of memory
alloc_tensor_range: failed to allocate CUDA3 buffer of size 101645909504
llama_model_load: error loading model: unable to allocate CUDA3 buffer

The total VRAM was sufficient. The split wasn’t. llama.cpp assigns the highest layer indices to the last device, so all 15 GPU-resident expert layers landed on one 40 GB card and asked it for 97 GB. Balancing this needs explicit -ot tensor placement across the four devices rather than the layer-count shortcut. Given that the fundamental limit is total memory bandwidth, the payoff for getting that regex right is small.

What this tells you about K3.1 Pro speed

Nothing numeric, and we will not pretend otherwise. Our 0.10 tokens per second belongs to a 1-bit K3 GGUF on a CPU-offload setup. No K3.1 GGUF exists to repeat it with, and no K3.1 recipe documents expert offload. What carries over is structural: K3.1 keeps the same 896 experts and 104B active parameters, so any future K3.1 offload setup will hit the same RAM-bandwidth wall, while a full eight-GPU node running vLLM or SGLang is a different machine class entirely. Until someone measures K3.1 on real hardware, any tokens-per-second figure you see for it is an estimate.

Is the output any good

Judging quality from a 1-bit quant running at a tenth of a token per second is not fair to the model, so we ran the same prompts against the hosted reference model instead. Moonshot trains and serves K3 with MXFP4 weights, which the size check above confirms for both releases, so even that isn’t BF16, but it’s the model behaving as its authors intend. Both answers were correct.

On a merge-overlapping-intervals problem it produced the standard sort-then-sweep solution at O(n log n), correctly treated touching intervals like [1,4] and [4,5] as mergeable, returned new lists rather than mutating the caller’s input, and listed empty input, single intervals, unsorted input and fully nested intervals as the cases it handled.

Given a systemd trap, asking why SSH dies after a reboot once ssh.socket is disabled on Ubuntu, it identified socket activation as the mechanism, explained that disabling the socket unit doesn’t enable ssh.service, and offered both the re-enable path and the explicit switch to a standalone daemon. That’s the correct answer to a question that catches plenty of administrators.

One behaviour will cost you money if you miss it. Kimi K3 defaults to a very high reasoning effort. At a 1200 token output cap our systemd question returned completely empty content, with all 1200 tokens consumed by hidden reasoning. Raising the cap to 4000 tokens returned empty again after 4088 reasoning tokens, billed past the cap rather than stopped at it. Only lowering the reasoning effort produced an answer, in 1116 tokens. Budget five thousand output tokens or more per call, or turn the reasoning effort down, otherwise you pay for thinking and receive nothing. K3.1 Pro’s card makes the same default explicit: reasoning_effort takes low, high or max, and max is the default.

Check the license before you deploy

Kimi K3.1 Pro ships under the K3 license with the product name changed. We diffed the two files and the thresholds are identical. It grants MIT-style rights to use, modify, fine-tune and sell, with two conditions that only apply to very large operators and one exemption:

• Model as a Service revenue above $20 million over any 12 months: a separate agreement with Moonshot first
• Products above 100 million monthly active users or $20 million monthly revenue: show “Kimi K3.1-Pro” prominently in the UI
• Internal use, and access through Moonshot’s own products or certified inference partners: exempt from both

For anyone running the weights on their own hardware for their own team, neither condition applies. The MaaS definition also excludes end-user products where the model sits behind specific features, so an internal coding assistant or a product feature is not a “service” under section 2. If you resell raw API access, read section 2 in the LICENSE file you downloaded earlier rather than this summary.

What it costs to run Kimi K3 locally

Owning hardware that runs this model well is not a homelab decision. Fitting all 594 GB in VRAM so generation isn’t bandwidth-starved means eight 80 GB cards at minimum, and Unsloth reports roughly 20 tokens per second on B200 class hardware. The densest workstation card you can order is the 96 GB RTX PRO 6000 Blackwell (paid link), and even the 1-bit quant needs seven of them. That is a rack, not a workstation, and K3.1 Pro at 1.56 TB needs about two and a half times that capacity again.

Renting is the honest comparison. Our four A100 box ran at $3.87 an hour and the whole exercise, download and build and benchmark included, cost $8.29. It produced a model too slow to hold a conversation with. A box large enough to be quick costs several times that per hour, and hosted Kimi K3 sits at $3 per million input tokens and $15 per million output. The break-even against an API is measured in months of continuous use, and that maths only works if you have a compliance reason to keep the weights in your own building. We rented through vast.ai, where per-second billing keeps an experiment like this cheap.

For most people the practical answer is to keep Kimi as an API model and run something smaller on your own metal. If you want K3.1 Pro itself without 1.6 TB of GPU memory, the Kimi K3.1 Pro API setup gets you there with an API key, and the Kimi CLI with OpenRouter reaches K3 the same way. A 30B class model on a single 24 GB card gives you interactive speed for real work, and a RTX 3090 (paid link) remains the cheapest way into that bracket. Our llama.cpp local LLM guide and Ollama setup both cover that path, and the open-source LLM comparison helps you pick the model. For serving at production concurrency rather than single-user chat, SGLang and vLLM are the engines to compare, and anyone specifying a machine for local inference should start from the local AI workstation build.