The 7B, 32B and 70B labels on AI models are not version numbers or marketing tiers, they are a straight count of how many parameters the model holds, measured in billions. A parameter is one of the adjustable weights the model learned during training, and the count is the single best shorthand for how much memory the model needs to run on your own PC.

Quick Answer

The B stands for billions of parameters: 7B is 7 billion, 70B is 70 billion. More parameters generally means a more capable model that also needs more memory. At Q4_K_M quantisation a 7B model needs about 4.4GB of VRAM and fits an 8GB GPU, while a 70B needs roughly 40GB.

What a parameter actually is

During training, the model adjusts billions of internal numbers until it can predict text well. Each of those numbers is a parameter. A 70B model has roughly ten times as many of these weights as a 7B, which gives it more room to store nuance, reasoning patterns and world knowledge. That extra capacity is why larger models usually answer harder questions more reliably, though training quality and data matter too, so a well-trained 7B can outperform a sloppy 13B.

The catch is that every parameter has to sit in memory while the model runs. That is what ties the parameter count directly to the GPU you need.

How parameter count maps to VRAM

The headline figure assumes a level of compression called quantisation. Most people run models quantised to roughly 4 bits per parameter (the Q4_K_M level), which keeps quality high while cutting memory use dramatically. At that level the rough memory picture looks like this:

  • 7B at Q4_K_M: about 4.4GB, comfortable on an 8GB GPU
  • 32B at Q4_K_M: roughly 20GB, needs a 24GB card
  • 70B at Q4_K_M: around 40GB, which means a high-VRAM card or splitting the load across two

Run a model at full 16-bit precision instead and those numbers roughly quadruple, which is why local users almost always reach for a quantised version.

Picking a size for your hardware

The practical question is not "which is best" but "which is the biggest model my GPU can hold". An 8GB card runs a 7B comfortably and leaves headroom for context. A 16GB card opens up larger 7B and some 13B models at higher precision. For 32B you want 24GB, and 70B realistically needs a 48GB-class setup. If you are buying a machine specifically to run models locally, the AI PC range at Evetech puts VRAM headroom at the centre of every configuration, which makes it a practical reference when matching hardware to model size.

System RAM matters too if you offload part of the model to the CPU, but that path is much slower, so VRAM remains the number to chase. For a broader look at well-balanced machines that double as everyday workstations, the PC best sellers cover the practical middle ground.

Frequently Asked Questions

Does a bigger model always mean better answers?

Usually, but not always. More parameters give more capacity for nuance and reasoning, yet a well-trained smaller model can beat a poorly trained larger one. Training data and quality matter alongside raw size.

What does quantisation do to the memory numbers?

Quantisation compresses each parameter to fewer bits. At Q4_K_M, a 7B drops to about 4.4GB instead of the much larger full-precision figure, which is what makes running these models on a normal GPU possible.

What GPU do I need for a 70B model?

Roughly 40GB of VRAM at Q4_K_M, so a 48GB-class card or two cards sharing the load. It is the most demanding of the three common sizes and not something an 8GB GPU can hold.

Can I run a 7B model on a normal gaming PC?

Yes. A 7B at Q4_K_M needs about 4.4GB of VRAM, so any 8GB GPU runs it comfortably with room left for context, making it the easiest entry point into local AI.

Want to run capable models on your own machine instead of paying per query? Match the parameter size to your VRAM, then browse the AI PC range at Evetech to find a build with the memory headroom your chosen model needs.