A language model is a huge table of numbers, and why AI needs lots of VRAM comes down to one thing: those numbers must sit in the graphics card's fast memory to run at a useful speed. Bigger models need more room, and that room is the VRAM on your GPU.
Quick Answer
Each parameter costs bytes. At 16-bit, a 7 billion parameter model needs roughly 14GB just for weights, while a 4-bit quantised copy fits in about 4 to 5GB. Add a few more gigabytes for the context window and working memory. For local models, 12GB is a workable start, 16GB is comfortable, and a card in that class often sits around R10,000 to R15,000.
🧮 The Maths of Model Size
Multiply parameters by bytes per parameter. A 7B model at 16-bit uses 2 bytes each, so about 14GB. A 13B model at 16-bit needs roughly 26GB, and a 70B model about 140GB, far beyond any single consumer card. This is why people run smaller models, or compress them, to fit a home GPU.
🗜️ Quantisation Shrinks the Bill
Quantisation stores each number with fewer bits. Four-bit versions cut the memory to about a quarter of 16-bit, with only a modest drop in quality for most chat and coding tasks. That brings a 7B or 8B model into 6GB of VRAM territory and a 13B model into about 8 to 10GB. Tools such as Ollama and LM Studio download quantised files ready to use.
📏 Context Takes Memory Too
The context window is the text the model remembers during a chat, and it lives in VRAM as a cache. Longer conversations and documents grow it. A model that fits at a short context may run out of space at 32,000 tokens, so leave 2 to 4GB spare. If the model does not fit, the software spills layers into system RAM and generation slows from dozens of tokens per second to a crawl.
🏠 Why Run Locally from South Africa
Prompts stay on your machine, nothing depends on an overseas service, and a cut fibre line does not stop your work. For a varsity student working on a thesis dataset, or a small Pretoria business handling client files, privacy and offline access are real reasons. NVIDIA cards remain the easiest to set up, so check the GeForce graphics cards, or study the top-selling GPUs and compare their memory sizes.
Frequently Asked Questions
Why do language models need so much VRAM?
Every parameter must be loaded for each word generated, and only the GPU's fast memory feeds that fast enough.
What is quantisation?
It stores weights using fewer bits, such as 4 instead of 16, so the model takes far less space with slightly lower accuracy.
Can I run DeepSeek on a 6GB card?
Small distilled versions in 4-bit can run, but the full flagship model needs far more memory than any home card provides.
Does the context window use VRAM too?
Yes. A longer conversation grows the cache, so keep a few gigabytes free beyond the model file.
What happens if the model does not fit?
Layers move to system RAM and speed drops sharply. Choose a smaller or more compressed model instead.
Ready to run AI models on your own PC?
See which Evetech graphics cards offer the memory your chosen model needs and build a private, local setup.