Can You Run a Local Coding Model on CPU and RAM Alone?

Spin up a small coding model on a machine with no graphics card and it works, right up until you load something bigger and the cursor stalls between every word. That gap is the whole story of running a local coding model on CPU and RAM alone. It is genuinely possible, but the moment your model and its context outgrow what fits comfortably, generation speed falls off a cliff that turns an instant reply into a slideshow.

Quick Answer

Yes, you can run a local coding model on CPU and RAM alone. Compact quantised models between 3B and 8B parameters stay usable on a modern multi-core CPU with 16 to 32GB of RAM. But a model that fits fully in GPU VRAM generates at 40 tokens per second or more, while the same job spilling into system memory or running CPU-only commonly collapses to single digits, often 2 to 3 tokens per second.

Why CPU-only works at all

Local inference is bound by memory bandwidth far more than raw compute. Generating each token means streaming the model's weights through the processor, so the limiter is how fast those weights move, not how many cores you have. A CPU with plenty of fast RAM can do this for a small enough model, and tools like llama.cpp and Ollama are built to run cleanly without any GPU at all. For light autocomplete, short code snippets and quick refactors on a 3B to 8B quantised model, a CPU-only setup is perfectly serviceable.

System RAM is also cheap and plentiful compared with VRAM. A desktop with 32GB or 64GB of RAM can hold model weights that would never fit on a mid-range graphics card, which is exactly why people try the CPU route in the first place.

Where it falls apart

The problem is throughput. System memory throughput runs far below what a GPU's dedicated memory path can deliver, so as soon as the model is large, the processor spends most of its time waiting on memory. Push a 13B or larger model onto CPU alone, or let a model overflow VRAM and spill into system RAM, and benchmarks routinely show throughput dropping into the low single digits. Two to three tokens per second is slow enough that you wait several seconds for a single function to appear, which kills the flow that makes a coding assistant worth running.

Context length makes it worse. A long prompt full of your codebase fills memory and slows every subsequent token, so the same model that felt fine on a one-line question crawls once you feed it a whole file. The trade-off is consistent: CPU-only buys you the ability to run a model at all, at the cost of the speed that makes it pleasant to use.

The practical middle ground

For serious local coding work, a dedicated GPU with enough VRAM to hold the whole model is the difference between a tool you reach for and one you abandon. If a model fits entirely in VRAM it stays fast, and even a modest discrete card with adequate memory beats a fast CPU once models grow past the smallest tiers. The AI PCs at Evetech are specced around that balance, and the best-selling desktops cover configurations with the memory headroom local models need.

Frequently Asked Questions

What model size is realistic CPU-only?

Models between 3B and 8B parameters at a sensible quantisation are the sweet spot for CPU and RAM alone. They stay responsive enough for autocomplete and short generations. Beyond that, speed drops sharply and a GPU becomes worth it.

How much RAM do I need?

For small coding models, 16GB covers the basics and 32GB leaves room for both the model and a useful context window without pressure. More RAM lets you load bigger models, but it does not fix the bandwidth bottleneck that limits speed.

Why is GPU so much faster if CPU has more RAM?

Because GPU memory bandwidth is far higher. Inference streams weights constantly, so the GPU's wider, faster memory path generates tokens many times quicker than a CPU reading from slower system RAM.

Is 2 to 3 tokens per second usable for coding?

Barely. At that rate you wait noticeably for each line, which breaks concentration. It is fine for occasional batch tasks you can leave running, but frustrating for interactive coding.