You ran ollama ps, saw 100 percent CPU, and the obvious move feels like installing the NVIDIA CUDA toolkit inside your Ubuntu WSL2 distro. Do not. That single step is the most common reason Ollama will not use the GPU in WSL2, because WSL2 does not want a Linux GPU driver at all. It borrows the GPU from the Windows host through a passthrough layer, and a second driver installed inside Linux fights with it.
Quick Answer
WSL2 exposes your NVIDIA GPU through the Windows host driver, not a Linux one. The fix is to install or update the Windows NVIDIA driver only, never a Linux driver or CUDA toolkit inside WSL2. The host driver must be reasonably current, then nvidia-smi should work inside WSL2 and Ollama will detect the card.
Why the Linux driver breaks it
In WSL2 the GPU is shared from Windows. NVIDIA publishes the CUDA-enabled libraries straight into the distro at /usr/lib/wsl/lib/, mounted there automatically when a supported Windows driver is present. That is the whole supply chain: Windows driver up top, WSL libraries underneath, your app on top of those. Install a separate Linux NVIDIA driver or full CUDA toolkit inside the distro and you stack a second, conflicting driver onto a passthrough that was already complete, which is exactly when GPU detection stops working. If you have built a local AI box around this, the AI PC range at Evetech shows the kind of NVIDIA hardware this passthrough is designed for.
The fix, in order
Work through these on the Windows side first, then check inside WSL2.
- Confirm you are on WSL2. In PowerShell run
wsl -l -v and check the VERSION column reads 2. WSL1 has no GPU passthrough at all. - Update the Windows NVIDIA driver to a current release from the host, not from inside Linux. A WSL2-capable driver is required, and recent Game Ready or Studio drivers all qualify.
- If you ever installed a Linux NVIDIA driver or CUDA toolkit inside the distro, remove it so only the WSL passthrough libraries remain.
- Inside WSL2 run
nvidia-smi. If it lists your card, the passthrough is healthy. - Reinstall or restart Ollama so it picks up CUDA, then run
ollama ps while a model is loaded. You should now see GPU usage instead of 100 percent CPU.
Order matters: get nvidia-smi working before you touch Ollama, because Ollama only finds the GPU once the layer beneath it is sound.
How to confirm it actually worked
nvidia-smi inside WSL2 is your first signal, it proves the host driver and passthrough are talking. The second is ollama ps during inference: a GPU-accelerated model reports GPU rather than pure CPU, and tokens stream noticeably faster. If nvidia-smi works but Ollama still sits on CPU, restart the Ollama service so it re-scans for CUDA on a clean start. Local LLM work leans hard on the card, and the most popular PC builds list is a quick way to gauge what configurations local builders are running inference on.
Frequently Asked Questions
Do I need to install CUDA inside WSL2 for Ollama?
No. The CUDA runtime libraries are mounted into WSL2 automatically once a supported Windows driver is installed. Adding a separate Linux CUDA toolkit is the step that usually breaks detection.
Why does nvidia-smi work but Ollama still uses CPU?
Ollama scans for CUDA at start-up. If the driver became healthy after Ollama launched, restart the service so it picks up the GPU on a fresh start.
What Windows version do I need?
Windows 11, or Windows 10 21H2 and later, with a WSL2-capable NVIDIA driver on the host. Older builds lack the passthrough plumbing entirely.
Does this apply to Docker on WSL2 as well?
The driver rule is the same, but containerised Ollama also needs the NVIDIA container toolkit registered as the Docker runtime inside WSL2. That is a separate step on top of a working host driver.
Running local models well starts with the right NVIDIA card and a clean driver setup. Explore the AI PC range at Evetech to build a machine that drives Ollama on the GPU from day one.