Quick Answer

Running local LLMs comes down to VRAM: a 24GB card runs a 13B model comfortably, while 16GB handles 7-8B models fast. An RTX 5080-class build does this well in Klerksdorp for around R40,000, and more system RAM extends what you can offload.

VRAM and quantisation set the model size

Local LLMs (Llama, Mistral, Qwen) load into VRAM. 16GB runs a 7-8B model at full speed; 24GB comfortably fits a 13B model, and quantised (4-bit) versions stretch this further. The RTX 5080 (16GB) or higher is the practical target. Tokens-per-second scale with memory bandwidth, so newer cards generate noticeably faster.

RAM for CPU offload and context

When a model exceeds VRAM, layers offload to system RAM - so 64GB DDR5 lets you run larger models slowly and keep big context windows. A fast Ryzen 7/9 helps CPU-offloaded layers. A 2TB Gen4 NVMe (from R2,200) stores multiple multi-gigabyte model files for quick switching.

Setup, delivery and Klerksdorp notes

Tools like Ollama or LM Studio make local models easy to run. North West maize belt makes good cooling worthwhile under sustained inference. On arrival, install the latest driver and benchmark tokens-per-second on a 7B model to set expectations.

FAQ

How much VRAM to run a local LLM?

16GB runs 7-8B models at full speed; 24GB comfortably fits a 13B model. Quantised 4-bit versions let larger models fit in less VRAM with a small quality trade-off.

Can I run a local LLM without a big GPU?

Yes, slowly - layers offload to system RAM, so 64GB DDR5 lets you run larger models on CPU. A strong GPU is far faster, but RAM offload is a workable budget path.

How fast is delivery to Klerksdorp?

Evetech reaches Klerksdorp in 1-3 working days from Joburg. Install Ollama or LM Studio on arrival and benchmark a 7B model to confirm tokens-per-second.

Run models locally, privately An RTX 5080-class GPU with 64GB DDR5 runs 13B models on your own machine in Klerksdorp. Browse Evetech's AI-ready PCs and start with Ollama to benchmark tokens-per-second on a 7B model.