FLUX.1 Dev at full FP16 is simply too large for an 8GB card, weighing in around 23GB before you even load the text encoder. The fix is GGUF quantisation: load a Q4 or Q5 build through the ComfyUI-GGUF nodes and the model shrinks to a size that fits comfortably on mainstream hardware. A Q4 build lands around 6 to 7GB, bringing Flux within reach of an RTX 4060-class GPU.

Quick Answer

Replace the full-precision Flux model with a Q4 or Q5 GGUF build loaded via the city96 ComfyUI-GGUF extension. Q4_K_S sits around 6.8GB and is the sweet spot for an 8GB card, leaving just enough headroom for computation, while Q5 preserves roughly 95 percent of the original quality. Pair it with the GGUF version of the T5 text encoder, not the FP16 one.

Why Flux will not fit at full precision

Diffusion models live in VRAM, and FLUX.1 Dev is a heavyweight. At full FP16 the model alone is roughly 23GB, far beyond what an 8GB card can hold, and that is before the T5 text encoder, which in FP16 form eats around 9GB on its own. Try to load that on an 8GB GPU and it either fails outright or offloads so heavily that generation becomes unusable. The model simply does not fit, so the answer is to make it smaller rather than to fight the overflow.

Quantisation compresses the numbers that make up the model so they occupy less memory. GGUF is the format that packages those compressed weights, and the ComfyUI-GGUF nodes let you load them directly in your existing workflow.

Choosing Q4 or Q5

The two practical choices for an 8GB card are Q4 and Q5, and the difference is a small quality-versus-headroom trade. Q4_K_S compresses Flux to around 6.8GB, which is the sweet spot on 8GB because it leaves room for the active computation. Q5_K_S is a touch larger but preserves roughly 95 percent of the original quality, with the loss barely noticeable next to the near-lossless Q8. Flux happens to be relatively quantisation-resistant, holding quality well even at lower bit depths, so even Q4 produces genuinely usable images.

The one catch worth knowing is prompt adherence. Q4 can be slightly less precise at following long, highly detailed prompts, while Q5 handles complex prompts more faithfully. If you have any headroom, Q5 is the better default; if you are right at the 8GB edge, Q4 keeps you stable. The AI PC range at Evetech shows what a comfortable image-generation machine looks like, and a card from the top-selling graphics lineup with more VRAM removes the constraint altogether.

Do not forget the text encoder

The mistake that trips people up is loading the full FP16 T5 encoder alongside the quantised model. At roughly 9GB, the FP16 T5 alone blows the budget on an 8GB card. Grab the GGUF version of the T5 encoder instead so both the model and the encoder are compressed, and add ComfyUI's low-VRAM flag so weights can stream from system RAM when needed. With both quantised and the right flag set, Flux runs on hardware that the full model would never touch.

Frequently Asked Questions

Why does FLUX.1 Dev not fit in 8GB?

At full FP16 the model is roughly 23GB, far more than 8GB can hold, and the FP16 text encoder adds around 9GB on top. Quantising both to GGUF shrinks them to a size that fits an 8GB card.

What is the best GGUF quant for an 8GB card?

Q4_K_S at around 6.8GB is the sweet spot, leaving headroom for computation. Q5_K_S is slightly larger but keeps about 95 percent of the quality, so use Q5 if you have the room and Q4 if you are at the edge.

How much quality do I lose with quantisation?

Less than you might expect. Flux is relatively quantisation-resistant, so Q5 keeps roughly 95 percent of the original quality and even Q4 produces usable images. The main difference shows in very detailed prompt adherence.

Do I need a special extension?

Yes, the city96 ComfyUI-GGUF extension provides the loader nodes that read GGUF models in ComfyUI. Install it, then point it at your Q4 or Q5 Flux build and the GGUF T5 encoder.

Why is my GGUF Flux still running out of memory?

Usually because the FP16 T5 encoder is loaded instead of the GGUF one, which alone uses around 9GB. Swap in the GGUF T5 and add ComfyUI's low-VRAM flag so weights stream from system RAM.

Want Flux running smoothly on a mainstream card? Compare the AI PCs at Evetech and choose a build with the VRAM headroom to run quantised, or full-precision, image models without the squeeze.