Quick Answer
Running local LLMs comes down to VRAM: a 24GB card runs a 13B model comfortably, while 16GB handles 7-8B models fast. An RTX 5080-class build does this well in Witbank for around R40,000, and more system RAM extends what you can offload.
VRAM and quantisation set the model size
Local LLMs (Llama, Mistral, Qwen) load into VRAM. 16GB runs a 7-8B model at full speed; 24GB comfortably fits a 13B model, and quantised (4-bit) versions stretch this further. The RTX 5080 (16GB) or higher is the practical target. Tokens-per-second scale with memory bandwidth, so newer cards generate noticeably faster.
RAM for CPU offload and context
When a model exceeds VRAM, layers offload to system RAM - so 64GB DDR5 lets you run larger models slowly and keep big context windows. A fast Ryzen 7/9 helps CPU-offloaded layers. A 2TB Gen4 NVMe (from R2,200) stores multiple multi-gigabyte model files for quick switching.
Setup, delivery and Witbank notes
Tools like Ollama or LM Studio make local models easy to run. Highveld dust makes good cooling worthwhile under sustained inference. Evetech ships to Witbank in 1-3 working days from Joburg. On arrival, install the latest driver and benchmark tokens-per-second on a 7B model to set expectations.
FAQ
How much VRAM to run a local LLM?
16GB runs 7-8B models at full speed; 24GB comfortably fits a 13B model. Quantised 4-bit versions let larger models fit in less VRAM with a small quality trade-off.
Can I run a local LLM without a big GPU?
Yes, slowly - layers offload to system RAM, so 64GB DDR5 lets you run larger models on CPU. A strong GPU is far faster, but RAM offload is a workable budget path.
How fast is delivery to Witbank?
Evetech reaches Witbank in 1-3 working days from Joburg. Install Ollama or LM Studio on arrival and benchmark a 7B model to confirm tokens-per-second.
Run models locally, privately
An RTX 5080-class GPU with 64GB DDR5 runs 13B models on your own machine in Witbank. Browse Evetech's AI-ready PCs and start with Ollama to benchmark tokens-per-second on a 7B model.