Setting up a running local LLMs rig in Richards Bay

Quick Answer

For running local LLMs in Richards Bay, prioritise VRAM over clock speed: an RTX 4060 Ti 16GB (~R12,000) runs 7B–13B models well, and 32GB of system RAM plus a fast SSD round out the build. A complete local-AI rig runs R25,000–R40,000.

VRAM is the deciding spec

Language models are sized by VRAM. A 4-bit 7B model fits in about 6GB, a 13B in 10–12GB, and bigger models need offloading or 24GB+ cards. That makes a 16GB card — the RTX 4060 Ti 16GB or RTX 4070 Ti Super 16GB — the value pick for Richards Bay buyers, because the moment a model overflows into system RAM your tokens-per-second drop sharply. Back it with a Ryzen 7 7700 and 32GB DDR5.

Storage, power and heat

LLM weights are big — a 2TB Gen4 NVMe (around 7,000 MB/s reads) loads a 9GB model in a second or two and holds several. Richards Bay's coastal humidity and heat make cooling matter: a mesh case with two intake fans and a 750W 80+ Gold PSU keep a 4070-class card stable during long inference runs.

Context length and RAM

Longer context windows (8k, 16k, 32k tokens) consume extra VRAM on top of the model itself, so leave headroom rather than maxing the model size. On a 16GB card, a 13B model at a moderate context is a comfortable target. System RAM of 32GB supports the OS and model loading without becoming a bottleneck.

FAQ

What's the minimum GPU for usable local LLMs?

An 8GB card runs small 7B models, but a 16GB card like the RTX 4060 Ti 16GB is far more flexible and only marginally pricier per token.

Do I need 64GB of system RAM?

Not for GPU inference — 32GB is plenty. You'd want 64GB+ only if you deliberately offload large models to system memory.

How much SSD space for several models?

A 2TB NVMe is sensible: quantised models range from 4GB to 40GB, so 2TB comfortably holds a working library.