Hardware delivery to Plettenberg Bay for running local LLMs

Quick Answer

Running local LLMs comes down to VRAM: a 24GB card runs a 13B model comfortably, while 16GB handles 7-8B models fast. An RTX 5080-class build does this well in Plettenberg Bay for around R40,000, and more system RAM extends what you can offload.

VRAM and quantisation set the model size

Local LLMs (Llama, Mistral, Qwen) load into VRAM. 16GB runs a 7-8B model at full speed; 24GB comfortably fits a 13B model, and quantised (4-bit) versions stretch this further. The RTX 5080 (16GB) or higher is the practical target. Tokens-per-second scale with memory bandwidth, so newer cards generate noticeably faster.

RAM for CPU offload and context

When a model exceeds VRAM, layers offload to system RAM - so 64GB DDR5 lets you run larger models slowly and keep big context windows. A fast Ryzen 7/9 helps CPU-offloaded layers. A 2TB Gen4 NVMe (from R2,200) stores multiple multi-gigabyte model files for quick switching.

Setup, delivery and Plettenberg Bay notes

Tools like Ollama or LM Studio make local models easy to run. holiday-town peak-season delays makes good cooling worthwhile under sustained inference. Evetech ships to Plettenberg Bay in 3-5 working days off the N2. On arrival, install the latest driver and benchmark tokens-per-second on a 7B model to set expectations.

FAQ

How much VRAM to run a local LLM?

16GB runs 7-8B models at full speed; 24GB comfortably fits a 13B model. Quantised 4-bit versions let larger models fit in less VRAM with a small quality trade-off.

Can I run a local LLM without a big GPU?

Yes, slowly - layers offload to system RAM, so 64GB DDR5 lets you run larger models on CPU. A strong GPU is far faster, but RAM offload is a workable budget path.

How fast is delivery to Plettenberg Bay?

Evetech reaches Plettenberg Bay in 3-5 working days off the N2. Install Ollama or LM Studio on arrival and benchmark a 7B model to confirm tokens-per-second.