Setting up a running local LLMs rig in eMalahleni

Quick Answer

A local-LLM rig in eMalahleni (Witbank) is decided by GPU VRAM: an RTX 4060 Ti 16 GB (~R10,500) runs quantised 7B-13B models like Llama 3 and Mistral smoothly, while a 24 GB RTX 4090 is needed for 70B models. Pair the GPU with a Ryzen 5 7600 and 32 GB system RAM. For local AI, VRAM size matters far more than gaming frame rates.

Why VRAM decides your local-LLM rig

Local large language models load into GPU memory, so VRAM caps the model size you can run fast:

• 8 GB: small 7B models heavily quantised (Q4), tight
• 12 GB: 7B-8B models comfortably (Llama 3 8B, Mistral 7B)
• 16 GB (RTX 4060 Ti 16 GB): 13B models and longer context windows
• 24 GB (RTX 4090): 70B quantised models, the prosumer ceiling

System RAM matters too - 32 GB lets you offload layers to CPU when a model is slightly too big for VRAM, though performance drops sharply when it does.

A practical local-LLM build

• GPU: RTX 4060 Ti 16 GB (~R10,500) - the value pick for 7B-13B
• CPU: Ryzen 5 7600 (~R4,200) - feeds the GPU and handles CPU offload
• RAM: 32 GB DDR5 (~R2,000)
• Storage: 1 TB NVMe (~R1,200) - model files run 4-40 GB each

eMalahleni's dusty Highveld air means a mesh case with washable filters keeps temps stable during long inference sessions. Evetech ships there in 2-3 business days.

FAQ

How much VRAM do I need to run a local LLM?

12 GB runs 7B-8B models well; 16 GB (RTX 4060 Ti 16 GB) handles 13B models with longer context. For 70B models you need 24 GB (RTX 4090) or multi-GPU. VRAM, not raw speed, sets your model ceiling.

Can I run Llama 3 locally on a R20,000 PC?

Yes. An RTX 4060 Ti 16 GB runs Llama 3 8B and Mistral 7B comfortably in tools like Ollama or LM Studio, generating tokens quickly at usable quality with Q4/Q5 quantisation.

Does Evetech deliver to eMalahleni?

Yes, Evetech ships to eMalahleni (Witbank) in 2-3 business days. Power-test the GPU and confirm it's detected before signing.