How to Run Flux Locally on Your Own GPU With ComfyUI

Generating images locally, with no cloud subscription eating into your budget, comes down to one number: VRAM. Running Flux locally through ComfyUI is very achievable, but the full FLUX.1 Dev checkpoint is roughly 23GB and has to load into video memory to work. Owners of a high-VRAM card can run it as-is, while everyone else uses a quantised GGUF build that shrinks the model to fit. This guide gets you from a fresh ComfyUI install to your first local Flux image.

Quick Answer

Run Flux locally in ComfyUI with either a 24GB GPU for the full FLUX.1 Dev checkpoint, or a quantised GGUF build on smaller cards. The ComfyUI-GGUF node extension loads Q4 to Q8 variants: Q4 fits in roughly 7GB and runs on 8GB GPUs, while Q8 is nearly indistinguishable from full quality at about half the VRAM.

Match the Flux build to your GPU

Pick the right version for your card before downloading anything.

• 24GB VRAM: the full FP16 checkpoint at roughly 23GB runs natively. Highest fidelity, no compromises.
• 12 to 16GB VRAM: the FP8 build at around 12GB, or a GGUF Q8 which is marginally smaller and virtually identical to full quality.
• 8GB VRAM: a GGUF Q4 build at roughly 7GB. Fine details and text are slightly softer, but composition and prompt adherence stay strong.

Anything below Q4, such as Q3 or Q2, starts losing anatomy and text rendering, so Q4 is the practical floor for FLUX.1 Dev. If your card is short on memory, the GPU listings at Evetech show which high-VRAM cards SA builders are choosing for local generation.

Step 1: Install ComfyUI

Download and set up ComfyUI for your platform and confirm it launches to its node graph in the browser. ComfyUI is the node-based interface that gives you fine control over the Flux pipeline, which is why it is the preferred front end for local generation. Get a blank workflow running first so you know the base install is healthy before adding Flux.

Step 2: Add the GGUF node extension

If you are running a quantised build, install the ComfyUI-GGUF custom node extension through the ComfyUI Manager. This is what lets ComfyUI load Q4 to Q8 GGUF files in place of the full checkpoint. Restart ComfyUI after installing so the new nodes register. Owners of a 24GB card running the full FP16 checkpoint can skip this step.

Step 3: Place the model files

Flux needs three pieces in the right folders:

1. The Flux model, either the full FP16 checkpoint or your chosen GGUF quant, in the models folder.
2. The text encoders, the CLIP and T5 files that turn your prompt into something the model understands.
3. The VAE, which decodes the result into a finished image.

Drop each file into its matching ComfyUI directory. A missing encoder or VAE is the most common reason a first run fails, so check all three are present before loading a workflow.

Step 4: Load a workflow and generate

Load a Flux text-to-image workflow. If you installed the GGUF extension, use the GGUF loader node to point at your quantised file instead of the standard checkpoint loader. Type a prompt, set your image size, and queue the run. The first generation is slower as the model loads into VRAM; subsequent runs are quicker because the weights stay resident.

If you run out of memory mid-generation, step down a quant level, for example from Q8 to Q4, or reduce the image resolution. For a machine built around serious local generation, the AI PC builds at Evetech combine strong GPUs with the VRAM headroom Flux demands.

Frequently Asked Questions

How much VRAM do I need to run Flux locally?

The full FLUX.1 Dev checkpoint is roughly 23GB and wants a 24GB GPU. A GGUF Q4 build drops that to about 7GB and runs on an 8GB card, while Q8 sits near full quality at roughly half the original VRAM.

What is the ComfyUI-GGUF extension for?

It lets ComfyUI load quantised GGUF versions of Flux, from Q4 up to Q8, instead of the full checkpoint. That is how owners of 8GB to 16GB cards run Flux without the 24GB the full model would otherwise require.

Is GGUF Q4 quality good enough?

For most work, yes. Q4 keeps composition and prompt adherence strong, with only slightly softer fine details and text compared to FP8 or FP16. It is the practical floor, since dropping to Q3 or Q2 starts degrading anatomy and text noticeably.

Why did my first Flux generation fail?

The usual cause is a missing file. Flux needs the model, the CLIP and T5 text encoders, and the VAE all placed in their correct ComfyUI folders. Check all three are present, and confirm the GGUF loader points at the right file if you are using a quant.

Q8 or FP8, which should I pick?

GGUF Q8 is marginally smaller in VRAM than FP8 and slightly higher fidelity, so it is a strong choice on a 12 to 16GB card. If you have a 24GB GPU, the full FP16 checkpoint gives the best quality of all.