Kimi K3 and DeepSeek V4 Flash split local AI into three setups

A laptop, USB drive, and server tower linked by three diverging paths. — A laptop, a USB drive, and a server tower split into three paths.Illustration: generated for this article

Running AI locally now splits into different jobs. A Kimi K3 model with 2.78 trillion parameters can run on 8.24GB of memory on a CPU without a GPU, and the kimi-k3-in-c GitHub project does that with less than 200KB of C code. That is one kind of local AI: storage-heavy, CPU-bound, and far removed from the desktop-app idea most people start with.

Another path looks very different. DeepSeek V4 Flash can run on a DGX Spark or on a Mac with 128GB of RAM, while the DeepSeek V4 Pro preview version reaches 1.6T total parameters. The DGX Spark's suggested retail price rose from $3999 to $4699 and up, which makes the hardware choice part of the workflow, not a footnote. The pattern here is simple: local AI now depends as much on deployment shape as on the model name.

Huge models can run on tiny RAM, but storage and latency turn that into a different class of workflow.

Kimi K3 shows why local AI is often a storage and bandwidth problem first. According to ifanr, the model has 2.78 trillion parameters, yet it can run on 8.24GB of memory on a CPU without a GPU. The catch is that the weights still occupy about 1.56TB of disk space.

The route to that tiny memory footprint is aggressive offload. ifanr says the model has 93 layers, 92 of them MoE, with 896 experts in each layer and 16 experts selected for each token. After the routed experts stay on disk, Kimi K3 still keeps 113.49GB of dense weights, and the project reorganized the 93 layers into one continuous file of about 109GB. The article frames the reduction path through 5.56TB, 1.56TB, 113.49GB, and 8.24GB.

That tradeoff is easy to miss if you only ask whether the model fits in RAM. In the minimum-memory configuration, Kimi K3 took 261.5 seconds to generate 8 tokens and averaged 32.69 seconds per token, or about 0.03 tokens per second. The test ran on a dual-socket AMD EPYC 7763 workstation with 124 CPU cores, 228GB of memory, and 3.2TB of NVMe SSD storage, with four NVIDIA L40 GPUs left unused.

DeepSeek points to the same split between size and practicality. ifanr says DeepSeek V4 Flash can run on a DGX Spark or on a Mac with 128GB of RAM, while the V4 Pro preview reaches 1.6T total parameters. The DGX Spark's suggested retail price also rose from $3999 to $4699 and up. For local deployment, the first question is VRAM capacity, followed by memory bandwidth.

Three Local-AI Workflows Compared Across Hardware, Runtime, and Deployment Style

Dimension	Kimi K3 via kimi-k3-in-c	BaseRT on Apple Silicon	NIGHTRUN via UEFI boot
Core deployment idea	Storage-heavy CPU inference for a huge MoE model	Optimized local inference engine for Apple Silicon	Bootable local large-model chat without an operating system
Target environment	CPU without a GPU	Apple M5 Pro with 48 GB memory	CPU offline; no GPU acceleration
Models mentioned	Kimi K3	Qwen, Llama, Gemma; tested on Qwen3, Qwen3.5, Qwen3.6, Llama 3.2, and Gemma 4	Llama 3.2 1B Instruct, Llama 3.2 3B Instruct, Granite 4.1 3B, and Qwen3 4B Instruct 2507
Scale / size noted	2.78 trillion parameters; official model card says 2.8 trillion parameters	Tested models covered parameter sizes from 0.6B to 35B	Model transferred into memory is between 1.3 and 2.4 GB
Performance claim	In the minimum-memory configuration, 261.5 seconds to generate 8 tokens; at the 8GB level, 32.69 seconds per token, or about 0.03 tokens per second	On an M5 Pro, prompt-processing speed reaches up to 6.4 times llama.cpp and 3.9 times MLX; generation speed reaches up to 1.75 times llama.cpp and 1.33 times MLX	Not covered
Representative benchmark	8.24GB of memory usage under a 8GB limit; peak RSS of 8.24GB	Qwen3.6-35B-A3B reached 110.7 tok/s in BaseRT, 63.1 tok/s in llama.cpp, and 101.9 tok/s in MLX; Qwen3-0.6B reached 530.8 tok/s in BaseRT, 386.1 tok/s in llama.cpp, and 398.1 tok/s in MLX	Not covered
Memory / storage footprint	Full weights occupy about 1.56TB of disk space; the reorganized main-branch file is about 109GB; the four key figures are 5.56TB, 1.56TB, 113.49GB, and 8.24GB	Not covered	Not covered
Implementation approach	Less than 200KB of C code; does not use PyTorch or CUDA; depends on a compiler, OpenMP, and the system math library	Command-line tooling for install, pull, chat, and serve	Installer generates a bootable image and writes it to a USB drive or SD card
Interface / access pattern	CPU inference workflow	Local chat session or API server	Chat prompt appears after booting from USB drive
OS relationship	Linux cgroup used to limit memory during testing	Not covered	No Linux, no operating system
Notable constraint or tradeoff	92 of 93 layers use MoE; each layer contains 896 experts, and the router selects 16 experts for each token	Performance was measured against llama.cpp and MLX on the same Apple hardware	Supports no GPU acceleration and is framed as a geek proof-of-concept

On Macs, runtime choice can matter more than model size.

On the M5 Pro with 48 GB memory, the same model can feel like a different product depending on the runtime. BaseRT is a local AI inference engine optimized for Apple Silicon, and appinn's comparison puts it ahead of both llama.cpp and MLX in the parts users feel first: prompt processing and generation.

That gap is not small. In prompt-processing tests, BaseRT reached up to 6.4 times llama.cpp and 3.9 times MLX on the M5 Pro. In generation speed, it reached up to 1.75 times llama.cpp and 1.33 times MLX. For Qwen3-35B-A3B, BaseRT decoded at 110.7 tok/s, while llama.cpp reached 63.1 tok/s and MLX 101.9 tok/s. A tiny Qwen3-0.6B model still showed the same pattern: 530.8 tok/s in BaseRT versus 386.1 tok/s in llama.cpp and 398.1 tok/s in MLX.

That matters because the local-AI argument is no longer only about capacity. The article's broader tension is throughput versus packing the model onto the machine in the first place, and on Mac hardware the runtime can swing the answer. BaseRT also ships with a simple path from install to use: `curl -LsSf https://basecompute.co/install.sh | sh`, then `basert pull Qwen/Qwen3-4B`, `basert chat Qwen/Qwen3-4B`, or `basert serve Qwen/Qwen3-4B --port 8080`. It exposes an OpenAI-compatible interface at `http://localhost:8080/v1`.

That turns Apple Silicon into a different kind of local-AI platform.

NIGHTRUN skips the operating system entirely, which changes what local AI is for.

NIGHTRUN pushes local AI into a different shape entirely. According to appinn, it is an LLM terminal that can boot directly via UEFI, write a bootable image to a USB drive or SD card, and then start a local large-model chat interface without an operating system. The developer frames it as a geek project meant to show that a local large model can live directly inside a UEFI firmware environment, and the source says the interface looks like a normal conversation window even though it behaves like an appliance.

That matters because the usual installation stack disappears. The model loads from a bootable USB drive, enters firmware mode, moves into memory, and then presents a chat prompt. Appinn says the model runs offline on the CPU, and that NIGHTRUN does not support GPU acceleration. The supported models are Llama 3.2 1B Instruct, Llama 3.2 3B Instruct, Granite 4.1 3B, and Qwen3 4B Instruct 2507.

This is why NIGHTRUN is not trying to be a general-purpose desktop app. It is closer to a local AI appliance: insert media, boot the machine, talk to the model. The source says the model transferred into memory is between 1.3 and 2.4 GB, which keeps the system in a small-model lane and makes the lack of drivers part of the design. Appinn says the developer wants AI chat to launch from a USB drive with no Linux and no operating system, and the article argues that adding GPU support would be hard because, without an operating system, there are no drivers. That tradeoff is the point.

How to choose a local AI deployment path

You need to make an extremely large MoE model usable on a CPU machine with no GPU, and you can tolerate very slow output. Use the storage-heavy CPU path shown by Kimi K3: the ledger says it can run on 8.24GB of memory on a CPU without a GPU, but at the 8GB level it averaged 32.69 seconds per token, or about 0.03 tokens per second. This is the right choice only when proving feasibility or fitting into very tight memory, not when you need interactive speed.
You want the same local model to feel fast on Apple Silicon and you care about both chat and serving an API. Use BaseRT on an Apple Silicon Mac. On an M5 Pro with 48 GB memory, it outperformed llama.cpp and MLX in the reported tests, including up to 6.4 times llama.cpp in prompt processing and 1.75 times llama.cpp in generation. It also supports both local chat and an OpenAI-compatible API server, so it fits desktop and service workflows.
You want to turn a machine into a self-contained appliance that boots straight into chat, with no operating system in the way. Use NIGHTRUN. The source says it boots directly via UEFI from a bootable USB drive, loads the model into memory, and then shows a chat prompt offline on the CPU. This is the most radical option, but it is also the least flexible because it does not support GPU acceleration.
You are trying to make an older Surface device into a practical daily Linux development machine before worrying about local-model inference. Follow the Fedora plus linux-surface path. The sources say Fedora Workstation was chosen for a Surface Pro 5 because of touch support and daily development use, and linux-surface provides the kernel patches, drivers, and related packages. This is the right choice when the deployment problem is hardware support and general usability, not model serving speed.

The practical choice is now deployment first, model second.

For day-to-day coding or agent use, the first question is not which model looks strongest. It is what kind of deployment the machine can sustain. According to ifanr, local setup starts with VRAM capacity and then memory bandwidth. That makes the practical split easy to see: a storage-heavy CPU route can make a huge model possible, a tuned Apple Silicon runtime can make a smaller model feel responsive, and an OpenAI-compatible local server can make the same model easy to wire into tools.

BaseRT fits that third case. According to appinn, it exposes an OpenAI-compatible interface at `http://localhost:8080/v1`, which is the kind of shape coding assistants and agent tools expect. For writing, that matters because the interface is predictable; for agents, it matters because they can call the service without learning a special client. The model is only part of the decision. The runtime and endpoint shape the work.

On ordinary laptops and tablets, the tradeoff is still real. sspai used a Surface Pro 5, a device that had been released for many years and was running Windows 10 22H2 before Fedora was installed. The writer chose Fedora Workstation because it fit a used Surface Pro 5 that needed both touch support and daily development use. Fedora 44 had recently been released, but the latest linux-surface-supported version was still 43 with kernel 6.19.8, so the article recommends downloading Fedora 43 first.

That setup shows the normal-OS path in plain terms. According to sspai, installation looked like a normal PC process: make a boot disk, enter UEFI, and adjust boot and Secure Boot settings. After that, the writer installed the linux-surface kernel, checked it with `uname -r`, and confirmed the surface identifier was running. Secure Boot can block linux-surface unless its MOK signing key is trusted, but the payoff is broad hardware support, including keyboard, touchpad, touchscreen, pen, Wi-Fi, Bluetooth, battery status, and suspend. For a real daily machine, that balance often beats novelty.

Before you start a development environment on Fedora, install linux-surface first. Add its repository, then let the package manager install and update the kernel. If Secure Boot is on, import and trust the MOK signing key or the new kernel may not boot.

Treat the Feature Matrix as your compatibility check, because linux-surface covers Surface Book and Surface Laptop too, not only Surface Pro models. On a Surface Pro 5, the basics are usually already in place, including keyboard, touchpad, touchscreen, pen, Wi-Fi, Bluetooth, battery status, and suspend. After rebooting, run `uname -r` and confirm the surface identifier is there.

For readers outside China

Availability: BaseRT is presented as an Apple Silicon-focused local inference engine; the ledger does not say whether it is available outside China. NIGHTRUN is a bootable UEFI-based local LLM terminal; the ledger does not say whether it is officially distributed outside China. The Fedora/linux-surface workflow is standard Linux software, but the ledger only documents it on a Surface Pro 5.
Pricing: The only price in the ledger is the DGX Spark suggested retail price, which rose from $3999 to $4699 and up. No pricing is disclosed for BaseRT, NIGHTRUN, Fedora, linux-surface, or the kimi-k3-in-c project.
Closest Western equivalents: BaseRT is closest in role to llama.cpp and MLX, since the ledger compares it directly against both.; NIGHTRUN is closest in spirit to a bootable appliance or firmware-level demo environment rather than a normal desktop app.; The Kimi K3 CPU path is closest to a minimal C-based inference implementation like kimi-k3-in-c, rather than a mainstream Python stack.
Data residency: NIGHTRUN runs offline on the CPU after booting from USB, so the ledger suggests a fully local flow with no OS layer, but it does not provide formal data-residency guarantees. BaseRT exposes a local OpenAI-compatible interface at http://localhost:8080/v1, which implies local processing, but the ledger does not specify telemetry or external network behavior. For Kimi K3, the ledger emphasizes local CPU inference and disk-heavy weights, but it does not discuss cloud fallback, logging, or data retention.

Sources

ifanr 8GB 内存也能跑 Kimi K3？2026 本地部署大模型配置全指南 https://ifanr.com/1673990
appinn BaseRT：专为 Apple Silicon 优化，让 Mac 本地大模型快 6.4 倍 https://appinn.com/basert
appinn NIGHTRUN - 无需操作系统，用 U 盘启动的的本地大模型客户端 https://appinn.com/nightrun
sspai 装上 Fedora 的 Surface Pro：一台老设备的新可能 https://sspai.com/post/112200

The evidence: 63 facts from 4 Chinese articles

Each line below was extracted from the article it sits under, in Chinese, before any of this was written. The writing is done from these and never from the source prose - that separation is structural, not a promise. How we work.

appinnBaseRT：专为 Apple Silicon 优化，让 Mac 本地大模型快 6.4 倍

BaseRT is a local AI inference engine optimized for Apple Silicon.
BaseRT can run open-source models such as Qwen, Llama, and Gemma.
On an M5 Pro, BaseRT's prompt-processing speed reaches up to 6.4 times llama.cpp and 3.9 times MLX.
On an M5 Pro, BaseRT's generation speed reaches up to 1.75 times llama.cpp and 1.33 times MLX.
BaseRT is installed with the command `curl -LsSf https://basecompute.co/install.sh | sh`.
A Qwen model can be downloaded with `basert pull Qwen/Qwen3-4B`.
A local chat session can be started with `basert chat Qwen/Qwen3-4B`.
An API server can be started with `basert serve Qwen/Qwen3-4B --port 8080`.
BaseRT provides an OpenAI-compatible interface at `http://localhost:8080/v1`.
The performance comparison used an Apple M5 Pro with 48 GB memory.
BaseRT was tested on Qwen3, Qwen3.5, Qwen3.6, Llama 3.2, and Gemma 4 models.
The tested models covered parameter sizes from 0.6B to 35B.
In decode speed tests, Qwen3.6-35B-A3B reached 110.7 tok/s in BaseRT, 63.1 tok/s in llama.cpp, and 101.9 tok/s in MLX.
In decode speed tests, Qwen3-0.6B reached 530.8 tok/s in BaseRT, 386.1 tok/s in llama.cpp, and 398.1 tok/s in MLX.
In prompt-processing tests, Qwen3-0.6B reached 31,183 in BaseRT pp2048 and 19,232 in llama.cpp pp2048.

appinnNIGHTRUN – 无需操作系统，用 U 盘启动的的本地大模型客户端

NIGHTRUN is an LLM terminal that can boot directly via UEFI.
NIGHTRUN can enter a local large-model chat interface immediately after starting the computer from a bootable USB drive, without an operating system.
The installation program generates a bootable image and writes it to a USB drive or SD card.
NIGHTRUN can boot from a USB drive, start its firmware function, load the model into memory, and then display a chat prompt.
The model runs offline on the CPU.
NIGHTRUN supports Llama 3.2 1B Instruct, Llama 3.2 3B Instruct, Granite 4.1 3B, and Qwen3 4B Instruct 2507.
NIGHTRUN does not support GPU acceleration.
The source says the model transferred into memory is between 1.3 and 2.4 GB.
The source frames NIGHTRUN as a system whose interface is simply a large-model conversation window, even though it functions like an operating system.

ifanr8GB 内存也能跑 Kimi K3？2026 本地部署大模型配置全指南

A Kimi K3 model can run on 8.24GB of memory on a CPU without a GPU.
DeepSeek V4 Flash can run on a DGX Spark or on a Mac with 128GB of RAM.
DeepSeek R1, which appeared in early 2025, triggered a wave of local-deployment tutorials.
Kimi K3 has 2.78 trillion parameters.
DeepSeek V4 Flash keeps 284B parameters, and the DeepSeek V4 Pro preview version has 1.6T total parameters.
The DGX Spark's suggested retail price rose from $3999 to $4699 and up.
The kimi-k3-in-c GitHub project uses less than 200KB of C code for Kimi K3 CPU inference.
The project does not use PyTorch or CUDA, and it only depends on a compiler, OpenMP, and the system math library.
Kimi K3's full weights still occupy about 1.56TB of disk space.
Kimi K3's official model card says the model has 2.8 trillion parameters, but each token activates about 104 billion parameters, or 3.7% of the total.
Kimi K3 has 93 layers, and 92 of them use MoE.
Each Kimi K3 layer contains 896 experts, and the router selects 16 experts for each token.
After the routed experts are left on disk, Kimi K3 still has 113.49GB of dense weights that cannot be avoided.
The project reorganized the 93 layers of dense weights into one continuous file of about 109GB.
The four key figures in the memory reduction path are 5.56TB, 1.56TB, 113.49GB, and 8.24GB.
Kimi K3 has 69 KDA layers, and KDA keeps a fixed-size recurrent state instead of a full KV cache for each historical token.
Kimi K3 also has 24 MLA layers that compress attention cache through low-dimensional representations.
In the minimum-memory configuration, Kimi K3 took 261.5 seconds to generate 8 tokens and reached 8.24GB of memory usage.
At the 8GB level, Kimi K3 averaged 32.69 seconds per token, or about 0.03 tokens per second.
The author's data came from a dual-socket AMD EPYC 7763 workstation with 124 CPU cores, 228GB of memory, and 3.2TB of NVMe SSD storage.
The workstation also had four NVIDIA L40 GPUs, but the test did not use the GPUs.
The author limited the process to 8GB of memory with Linux cgroup and measured a peak RSS of 8.24GB.
For local deployment, the article says the first thing to look at is VRAM capacity, followed by memory bandwidth.

sspai装上 Fedora 的 Surface Pro：一台老设备的新可能

The writer used a Surface Pro 5, which had been released for many years, as the device for this Linux experiment.
The Surface Pro 5 was running Windows 10 22H2 before Fedora was installed.
The writer discovered the linux-surface project and became curious whether installing an actively updated Linux distribution would make the Surface Pro 5 more usable.
Fedora Workstation usually follows new releases of GNOME, Wayland, and the Linux kernel early, and each Fedora version is maintained for about 13 months.
The writer chose Fedora Workstation because it was a good fit for a used Surface Pro 5 that needed both touch support and daily development use.
Fedora 44 had recently been released at the time of writing, while the latest linux-surface-supported version was still 43 with kernel 6.19.8.
The article recommends downloading the Fedora 43 image first if installing Fedora on a Surface at that time.
The Fedora installation process on the Surface Pro 5 was described as similar to that of a normal PC, including making a boot disk, entering UEFI, and adjusting boot and Secure Boot settings.
After installing Fedora, the writer recommends installing the linux-surface kernel before creating a development environment.
linux-surface maintains patches, drivers, and related packages for the Linux kernel for Surface devices.
The official linux-surface installation method is to add the appropriate repository and then install and update the kernel through the system package manager.
Secure Boot can prevent linux-surface from booting unless its MOK signing key is imported and trusted.
The linux-surface Feature Matrix covers multiple device lines, including Surface Book and Surface Laptop, not only Surface Pro models.
For an older Intel model such as Surface Pro 5, keyboard, touchpad, touchscreen, pen, Wi-Fi, Bluetooth, battery status, and suspend are usually already fairly complete under linux-surface.
After installation, the writer checked the kernel version with uname -r and confirmed that a version with the surface identifier was running.
The writer says the Surface Pro 5 was then running standard Fedora together with an extra Surface-adapted kernel, while continuing to use Fedora's normal package management.