What Are the Hardware Requirements to Run Ollama Locally?

The hardware requirements to run Ollama locally come down to one number: how much memory your model needs to sit entirely on the GPU. Ollama itself is light, but the model you load decides everything. A small 7B model fits comfortably on an 8GB card, while a 70B model demands workstation-class memory or accepts a painful slowdown as it spills onto system RAM.

Quick Answer

A 7B model at Q4 quantisation needs roughly 4 to 6GB of VRAM and runs well on an 8GB GPU. A 13B model wants 8 to 10GB, and 70B models realistically need 48GB or more. Below the GPU, plan on at least 16GB system RAM and an NVMe SSD.

Match the Model to Your VRAM

The single biggest factor is video memory. Ollama runs fastest when the entire model lives in VRAM, because the moment it overflows to system RAM the speed collapses to a crawl. A rough rule for Q4_K_M quantisation is about 0.55GB per billion parameters, plus overhead, so the model size you can run scales almost linearly with the card you own.

Using the common Q4 quantisation as the baseline: a 7B model needs 4 to 6GB and is happy on an 8GB card; a 13B model needs 8 to 10GB and wants a 12GB to 16GB card; a 32B model needs around 20GB; and a 70B model needs 48GB or more. A 16GB card is the practical sweet spot, comfortably handling 7B to 14B models at a responsive pace.

Quantisation Changes the Math

Quantisation is how a model is compressed to use less memory, and it has a dramatic effect on what fits. Q4_K_M is the efficiency standard for consumer hardware, trading a small accuracy loss for roughly a third of the memory a full-precision model would need. Step up to Q8 and memory use roughly doubles; FP16 doubles it again. If a model is just over your VRAM budget, a lower quantisation often brings it back into range. The pre-built AI PCs at Evetech are specced around exactly this kind of GPU-first local inference work.

Do Not Forget RAM, CPU and Storage

VRAM gets the attention, but the rest of the system matters. Treat 16GB of system RAM as the floor, partly because models load through RAM and partly so the machine stays usable while inference runs. The CPU is less critical when the GPU does the heavy lifting, though it carries the load on CPU-only setups. Storage should be an NVMe SSD: model files are several gigabytes each, and a fast drive shortens the wait every time you switch models.

A Sensible Buying Ladder

If you are buying for local AI, think in VRAM tiers. 8GB covers the essential 7B models for chat and coding help. 12GB adds breathing room. 16GB to 24GB opens up 30B-class models. 48GB is the entry to true 70B inference. Pick the tier that matches the models you actually intend to run rather than the largest model in existence. The current GPU and prebuilt options worth comparing show up in the Evetech PC best sellers list.

Frequently Asked Questions

How much VRAM do I need for a 7B model?

About 4 to 6GB at Q4 quantisation, which fits comfortably on an 8GB GPU. That covers most popular small models used for chat, summarising and coding assistance.

Can Ollama run without a GPU?

Yes, Ollama can run on CPU and system RAM alone, but it is much slower. Small models stay usable for light tasks; anything 13B or larger becomes sluggish without a capable GPU.

How much system RAM should I have?

Treat 16GB as the minimum. Models load through RAM before reaching the GPU, and the extra headroom keeps the rest of the machine responsive while a model is running.

What happens if a model is bigger than my VRAM?

Ollama offloads the overflow to system RAM, and performance drops sharply. Either choose a smaller model, use a lower quantisation level, or move to a card with more memory.

What runs a 70B model locally?

Realistically, 48GB of VRAM or more, which means a workstation-class card or a multi-GPU setup. Anything less forces heavy offloading to system RAM and very slow responses.