Ollama Falls Back to CPU After Sleep? Only a Full Reboot Fixes It

Your local model was flying on the GPU last night. This morning, after the machine woke from sleep, the same prompt crawls and your fans barely spin up. Restarting the Ollama service does nothing. The model is quietly running on the CPU, and the only thing that reliably brings the GPU back is a full reboot of the host.

Quick Answer

After a sleep or suspend cycle, Ollama often loses sight of the GPU and silently falls back to CPU inference. Restarting just the Ollama server process usually does not recover it, because the problem sits below Ollama in the GPU driver layer. A complete host reboot is the dependable fix that restores GPU detection and full speed.

What is actually going wrong

Ollama itself has not crashed. The break happens in the NVIDIA driver state that survives, or fails to survive, the suspend and resume cycle. When the system sleeps, the GPU memory management layer can come back in a state Ollama can no longer hand work to. Ollama detects no usable GPU, shrugs, and loads the model into system RAM on the CPU instead. Everything keeps working, which is exactly why it is easy to miss. You only notice because tokens that streamed instantly now arrive one slow word at a time.

Why restarting the service is not enough

The instinct is to restart Ollama, since that fixes most local-LLM hiccups. Here it usually fails. The Ollama process re-reads the GPU state on startup, but the underlying driver layer is still in its broken post-sleep condition, so the fresh process sees the same nonfunctional GPU and falls back to CPU all over again. You can restart the server five times and get the same crawling speed each time. That is the tell that this is a driver-state problem, not an application problem.

The reliable fix: reboot the host

A full reboot tears down and rebuilds the entire driver stack from scratch, which is why it works every time. Once the machine is back up, load a model and confirm the GPU is doing the work rather than assuming it is. If you load a model and it responds at full speed with the fans spinning up, you are back on the GPU. If it still crawls, the reboot did not take and something else is holding the driver, but in the overwhelming majority of cases one clean reboot is all it takes.

How to stop it happening daily

If you run local models on a machine you use every day, the fix is to stop the machine sleeping while it is your inference host. Disable sleep and suspend on a dedicated AI box, or get into the habit of a reboot rather than a wake whenever you have left it idle. A machine purpose-built for steady local inference, with enough VRAM to hold your model and a driver setup that stays put, sidesteps the whole cycle. The AI PC range at Evetech covers configurations designed for always-ready GPU workloads, and the broader PC best sellers list shows what SA buyers are pairing with their local model setups.

Frequently Asked Questions

How do I confirm Ollama is on the GPU and not the CPU?

Load a model and watch the response speed and the fans. GPU inference streams tokens quickly and spins the GPU fans up under load. If the reply trickles out and the card stays cold, you are on the CPU. Ollama's own process listing will also show where the model is loaded.

Will a driver update fix the sleep fallback?

Sometimes a newer GPU driver behaves better through suspend and resume, so keeping drivers current is worth doing. It is not guaranteed, though, which is why a reboot remains the dependable recovery when it happens.

Does this affect inference quality or just speed?

Just speed. The model produces the same output on CPU as on GPU, it simply runs far slower because the CPU lacks the parallel throughput a GPU brings to inference.

Can I just avoid sleeping the machine?

Yes, and that is the cleanest prevention. Turning off sleep on a machine you use as a local-model host sidesteps the GPU detection loss entirely, so you never have to reboot to recover it.