How to Run Flux on 8GB VRAM Using GGUF Quantised Models

FLUX produces some of the best local image generation around, and at full size it is far too big for an 8GB card. Quantisation closes that gap. GGUF quantised models compress FLUX from a roughly 23GB original down to around 5-7GB, small enough to run on an 8GB card like the RTX 4060. The trick is loading the GGUF version through the right ComfyUI nodes and picking the quantisation level that fits your card with a little room to spare.

Quick Answer

GGUF quantisation shrinks FLUX to roughly 5-7GB, letting 8GB cards such as the RTX 4060 run it. Load a FLUX GGUF model through city96's ComfyUI-GGUF loader nodes, and choose Q4 for the tightest fit or Q5 for better quality with a little more memory used. Add ComfyUI's --lowvram flag so weights stream from system RAM, and an 8GB card handles FLUX comfortably.

What GGUF quantisation actually does

The weights of a quantised model are stored in fewer bits than the original, reducing file size and VRAM footprint while keeping the model usable. GGUF is the format that packages these quantised weights for easy loading. For FLUX, that takes a model that would not fit on a consumer card and brings it into 8GB range. The quality cost is real but small at the levels you will use: Q4 images show slightly softer fine detail and marginally less accurate text, but composition and prompt adherence hold up well, and Q5 preserves around 95 percent of the original quality.

Step 1: Install ComfyUI and the GGUF nodes

Start with a working ComfyUI install, which is the interface this setup is built around. Then add the GGUF loader nodes: city96's ComfyUI-GGUF extension provides the custom nodes that let ComfyUI read and load GGUF-format models, which the standard loaders cannot do. Install it through ComfyUI's manager or by placing the extension in the custom nodes folder, then restart ComfyUI so the new nodes appear. Without this extension, ComfyUI has no way to load a GGUF file at all.

Step 2: Download the right FLUX GGUF model

Get a FLUX GGUF build at a quantisation level suited to 8GB. A FLUX.1-dev or FLUX.1-schnell GGUF at Q4 lands around 6.8GB, which fits an 8GB card with the streaming flag in place. If you want a lighter, faster option, the smaller distilled FLUX variants in GGUF form are only a couple of gigabytes and leave plenty of headroom. Match the file to your card: the more VRAM you have, the higher the quantisation level and resolution you can run.

Step 3: Choose Q4 or Q5

This is the key decision. Q4 is the tightest fit, the safest choice for a card that is right at the 8GB line, and it still produces genuinely usable images. Q5 keeps more quality, preserving close to 95 percent of the original, at the cost of a little more memory and a slightly tighter fit. On an 8GB card, start with Q4 if you want certainty, or try Q5 if you have any headroom and care about fine detail and text. There is no single right answer; it is a fit-versus-quality trade you tune to your card.

Step 4: Build the workflow with the GGUF loader

In ComfyUI, replace the standard model loader in your FLUX workflow with the GGUF Unet Loader node from the city96 extension, and point it at your downloaded GGUF file. Wire it into the rest of a normal FLUX workflow, your text encoders, sampler, and VAE decode. The GGUF loader handles the quantised weights; everything downstream behaves like a regular FLUX graph.

Step 5: Add the --lowvram flag

Launch ComfyUI with the --lowvram flag. This lets the model weights stream from system RAM rather than demanding all of them in VRAM at once, which is what keeps an 8GB card from overflowing on a model this size. It adds some overhead, but it is the difference between FLUX running and FLUX crashing on a tight card. With the GGUF model loaded and weights streaming, generation runs within your 8GB budget.

Step 6: Generate and tune

Run your first image and watch VRAM usage. If you have headroom to spare, try stepping up from Q4 to Q5 or nudging resolution higher. If you hit memory errors, drop back to a lower quantisation or keep resolution modest. The RTX 4060's newer architecture makes it quick per step even on 8GB, so iteration is faster than the raw VRAM figure suggests. If you find yourself constantly pinned against the memory limit, a card with more VRAM opens up higher quantisation and larger images, and the GPU range at Evetech lets you compare the price and VRAM of the upgrade options. For a complete machine specced for local AI generation, the AI PCs at Evetech take the guesswork out of matching card, memory and storage.

Frequently Asked Questions

Can an RTX 4060 really run FLUX?

Yes. A FLUX GGUF model at Q4 sits around 6.8GB, which fits the RTX 4060's 8GB with ComfyUI's --lowvram flag streaming weights from system RAM. The card's newer architecture also keeps per-step generation reasonably quick.

What is the difference between Q4 and Q5?

Q4 is more compressed, fits tighter memory, and still produces usable images with slightly softer detail. Q5 keeps closer to 95 percent of the original quality at the cost of a little more memory. On 8GB, Q4 is the safe choice and Q5 is worth trying if you have headroom.

Do I need ComfyUI specifically for GGUF FLUX?

This setup is built around ComfyUI with city96's ComfyUI-GGUF nodes, which are the standard way to load GGUF FLUX models. Without those nodes, ComfyUI cannot read the GGUF format, so they are essential to the workflow.

How much quality do I lose with quantisation?

Less than you might expect. Q4 shows slightly softer fine detail and marginally weaker text rendering, while composition and prompt adherence stay strong. Q5 is very close to the original. For most use the trade is well worth fitting FLUX onto an 8GB card.

Why do I need the --lowvram flag?

It lets the model weights stream from system RAM instead of all loading into VRAM at once, which keeps an 8GB card from overflowing on a model this size. It adds some overhead but is what makes FLUX run reliably on a tight card.