Setting up a running local LLMs rig in Witbank SA practical guide

Quick Answer

A local-LLM rig in Witbank (eMalahleni) is sized by GPU VRAM: an RTX 4060 Ti 16 GB (~R10,500) runs quantised 7B-13B models like Llama 3 and Mistral well, and a 24 GB RTX 4090 handles 70B models. Pair the GPU with a Ryzen 5 7600 and 32 GB system RAM. For local AI, VRAM capacity sets the model ceiling, not gaming frame rates.

VRAM tiers for local LLMs

Local models load into GPU memory, so VRAM caps the model size:

• 12 GB: 7B-8B models (Mistral 7B, Llama 3 8B)
• 16 GB (RTX 4060 Ti 16 GB): 13B models, longer context windows
• 24 GB (RTX 4090): 70B quantised models

32 GB system RAM lets you offload spillover to the CPU, though speed drops sharply when a model overflows VRAM, so match the GPU to your largest planned model.

A Witbank local-LLM build

• GPU: RTX 4060 Ti 16 GB (~R10,500)
• CPU: Ryzen 5 7600 (~R4,200)
• RAM: 32 GB DDR5 (~R2,000)
• Storage: 1 TB NVMe (~R1,200) - model files are 4-40 GB each

Witbank's dusty Highveld air makes washable dust filters important so the GPU stays cool during long inference runs. Evetech ships to Witbank (eMalahleni) in 2-3 business days; power-test before signing.

FAQ

What's the entry-level GPU for local LLMs?

A 12 GB card runs 7B-8B models, but the RTX 4060 Ti 16 GB (~R10,500) is the better value pick - its 16 GB handles 13B models and longer context, which extends the rig's usefulness as models grow.

Can I use this rig for both LLMs and Stable Diffusion?

Yes. The RTX 4060 Ti 16 GB runs both local LLMs (7B-13B) and Stable Diffusion (SDXL, Flux). The 16 GB VRAM is the common requirement for serious local-AI work across both.

Does Evetech deliver to Witbank?

Yes, Evetech ships to Witbank (eMalahleni) in 2-3 business days. Power-test the GPU and confirm it's detected before signing.