Your 70B model won't load on your Mac and the error feels like a hardware betrayal: you bought 48GB of unified memory, the model file is only 40-something gigs, so why the crash? The answer is that macOS never hands the whole pool to the GPU. It reserves a chunk for the system, and what is left falls short of what a 70B model at Q4 actually needs to sit in memory and run.

Quick Answer

Running a 70B model at Q4_K_M demands roughly 40 to 48GB sitting resident in memory, but macOS restricts GPU access to around 75 percent of total RAM on machines with 64GB or more, and closer to two thirds below that threshold. The practical minimum for success is 64GB. Machines with 32GB or 48GB run out of addressable memory before the model can even finish loading.

Why the headline RAM number lies

Apple Silicon integrates the processor and graphics cores onto one chip, meaning they share a single unified memory pool rather than maintaining separate banks. That sounds like a win until you learn the GPU cannot touch all of it. The Metal driver caps how much is addressable as GPU memory: about 75 percent on a 64GB-plus Mac, roughly two thirds on smaller configs. On a 64GB machine that is around 51GB available to the model, and you can nudge it higher with a terminal command, but you cannot reclaim the lot.

On top of that, macOS and your open apps quietly hold 4 to 6GB at all times, even with everything visibly closed. So a 48GB Mac is not offering 48GB to your model. It is offering somewhere in the low 30s once the system takes its share, and a 70B Q4 build simply does not fit. The result is swapping to SSD and a load that either crawls or dies.

What actually runs, and the SA alternative

If running 70B locally is the goal and you are set on a Mac, 64GB is the entry point and a chip in the M-series Max tier handles Llama-class 70B at Q4 comfortably with headroom for context. Step down to 48GB and you are forced into smaller models or heavier quantisation that hurts quality.

That said, plenty of SA developers get more local-AI performance per Rand from a desktop with a discrete GPU, where 24GB or dual-card VRAM setups chew through large models without the unified-memory tax. If you are weighing where your next AI machine should live, running the numbers against a dedicated local-AI desktop is often illuminating. GPU-first desktop builds with Rand pricing can be compared in the AI PC range at Evetech, while the PC best sellers page reflects what SA builders are actually choosing right now.

Frequently Asked Questions

Can I run a 70B model on a 48GB Mac at all?

Not the standard Q4_K_M build. After the GPU memory cap and system overhead you are left short of the 40-plus gigs it needs, so it swaps to disk or fails to load. A heavier quantisation may squeeze in, but with a quality hit.

Does the terminal command to raise GPU memory fix it?

It helps a little. You can push the addressable share higher than the default, but you still cannot give the GPU all your RAM, and the system overhead remains. It turns a borderline case into a usable one, not a 48GB machine into a 70B machine.

Is 64GB enough or should I go higher?

64GB is the comfortable floor for a single 70B model at Q4. If you want long context windows, multiple models loaded, or higher-precision quants, more memory keeps you out of swap.

Would a Windows PC with a GPU run 70B more easily?

Often yes, for the money. Discrete-GPU desktops avoid the unified-memory cap and let you scale VRAM, which is why many local-AI builders in SA go that route rather than a high-memory Mac.

Picking hardware for local LLMs? Match the memory to the model before you buy. Compare high-memory options in the AI PC range at Evetech and run your 70B models without the out-of-memory wall.