On Linux, Ollama requires NVIDIA driver 535 or later and bundles its own CUDA libraries internally, so installing a separate CUDA toolkit is both unnecessary and a common cause of detection failures. Remove any conflicting CUDA packages and install only the driver.
You install Ollama on a fresh Linux box, run a model, and watch your CPU pin to 100 percent while the GPU sits idle. The instinct is to go install the full CUDA toolkit, and that is usually the move that keeps Ollama stuck on the processor. On Linux, Ollama ships its own CUDA libraries internally and only needs a recent NVIDIA driver underneath it.
For GPU inference on Linux, Ollama needs NVIDIA driver 535 or newer, and nothing else from NVIDIA. It bundles the CUDA runtime it depends on, so a separately installed CUDA toolkit adds no benefit and can cause version conflicts that push inference back onto the CPU. Install the driver, confirm nvidia-smi works, then reinstall Ollama if it was set up before the driver.
Ollama bundles the exact CUDA libraries it was built against. When you install a standalone CUDA toolkit, you introduce a second set of CUDA libraries on the system, and library path conflicts can leave Ollama unable to load the GPU backend cleanly. Rather than crash, it falls back to CPU and keeps running, which is why the failure looks silent. People then assume the GPU is broken and pile on more CUDA packages, making the path collisions worse.
The driver is a different thing entirely. The NVIDIA kernel driver is what lets any CUDA software talk to the card at all, and Ollama needs version 535 or later for current GPUs. That single driver is the only NVIDIA component you must install yourself.
The AI PC range at Evetech brings together GPU-first configurations purpose-built for local AI inference workloads.
Start by confirming the driver is loaded. Run nvidia-smi: if it prints your card, its memory, and a driver version of 535 or higher, the driver layer is healthy. If the version is older, update it through your distribution's driver tooling and reboot.
Next, remove any standalone CUDA toolkit you added while troubleshooting. On Debian and Ubuntu systems that means purging the cuda packages so only the driver remains. You do not need nvcc or the developer toolkit for Ollama to run.
Finally, reinstall Ollama. If Ollama was installed before the driver was working, it never registered that a GPU exists, and a reinstall lets it detect the card on a clean system. Set OLLAMA_DEBUG=1 and restart the server, then check the startup log: a healthy boot reports an inference compute line naming the CUDA library and your GPU. If you see that line, the model is running on the card.
A driver that loads is not proof the model is on the GPU. Run a model and watch nvidia-smi in a second terminal; you should see VRAM fill and GPU utilisation climb while a prompt generates. If VRAM stays near zero, the model may be larger than your card's memory and Ollama is splitting layers to system RAM, which is slower but still partly accelerated. Picking a model that fits your VRAM keeps everything on the card. The PC best sellers at Evetech reveal which hardware local-AI builders in SA are currently pairing with inference workloads.
No, not for running Ollama itself. The toolkit is for compiling your own CUDA code. Ollama carries the CUDA runtime it needs, so the driver alone is enough for inference.
Usually a conflicting standalone CUDA install, or Ollama was installed before the driver was set up. Purge the extra CUDA packages and reinstall Ollama so it detects the card on a clean system.
NVIDIA driver 535 or newer on Linux. Older drivers can leave the GPU backend unavailable, sending inference to the CPU. Check the version at the top of nvidia-smi output.
Run a prompt and watch nvidia-smi: VRAM usage and GPU utilisation should rise. You can also set OLLAMA_DEBUG=1 and look for an inference compute log line naming the CUDA library.
If your current rig keeps falling back to CPU because the card is too small for the models you want to run, it may be time to build around the GPU instead. Explore the AI PC range at Evetech (https://www.evetech.co.za/PC-Components/ai-pcs-445) and spec a machine that keeps local inference where it belongs.
No, not for running Ollama itself. The toolkit is for compiling your own CUDA code. Ollama carries the CUDA runtime it needs, so the driver alone is enough for inference.
Usually a conflicting standalone CUDA install, or Ollama was installed before the driver was set up. Purge the extra CUDA packages and reinstall Ollama so it detects the card on a clean system.
NVIDIA driver 535 or newer on Linux. Older drivers can leave the GPU backend unavailable, sending inference to the CPU. Check the version at the top of nvidia-smi output.
Run a prompt and watch nvidia-smi: VRAM usage and GPU utilisation should rise. You can also set OLLAMA_DEBUG=1 and look for an inference compute log line naming the CUDA library.
How to fix low FPS after a USB DAC upgrade starts with audio drivers, USB bandwidth, and sample-rate checks. Re-test games with other ports, exclusive mode off, and background audio tools closed before blaming graphics hardware.
Read more
8000Hz polling is overrated for most SA gamers because it can add system load without a clear feel upgrade. Check FPS consistency, mouse sensor quality, USB stability, and the games you play before chasing the highest rate.
Read more
Intel Core Ultra can struggle in SA gaming benchmarks when power limits, game engines, or GPU pairing hide its strengths. Rustenburg gamers should compare cooling, resolution, BIOS settings, and workload mix first.
Read more
SA gpu price pressure questions need cautious SA context because pricing can move with availability, exchange rates, specs, and warranty support. Treat the trend as a planning signal, not a fixed price promise.
Read more
Second-hand GPUs are risky in SA because used cards can hide wear, heat damage, weak proof, or unstable performance. SA buyers should inspect condition, seller proof, return terms, and power needs before choosing a GPU.
Read more
Magnetic switches are rising in SA gaming keyboards because adjustable actuation can help fast movement and precise inputs. Polokwane gamers should compare rapid trigger, layout, software, noise, and switch feel before buying.
Read more
Black Friday Gaming PC Wait can help if your PC need is flexible. On a R20,000 SA budget, compare deal risk, component balance, warranty, and timing before waiting.
Pre-Black Friday webcam is a cautious guide for seasonal tech deal planning. Compare spec priorities, timing, warranty support, and realistic SA price checks for SA buyers without assuming live prices, availability, or exact benchmark results.
Choosing the best car phone holder mounting type is crucial for safety. Compare windshield, dashboard, and air vent mounts to find the perfect fit for your driving style and vehicle interior. 🚗📱
Wi-Fi 7 vs Wi-Fi 6e for cinematic story games comes down to router support, latency goals, device compatibility, and coverage. SA buyers should match the choice to the real network setup.
Top camera drones for video shoot 4K/60 in 10-bit colour profiles like D-Log M, capturing the dynamic range needed for professional-grade grading across SA landscapes, coastlines and travel content. The right choice depends on your sensor size priority, weight limit
For threadripper vs ryzen 9 for 3d artists, the Threadripper 9980X is the fastest tested CPU in Blender Cycles by a wide margin. For most 3D artists though, a Ryzen 9 remains the smarter investment before scaling into dedicated render nodes.
A UPS conditions mains power and gives battery runtime for a clean shutdown during an outage - protecting in-flight writes and ZFS pools that must never lose power mid-write. For any 24/7 server with data you value, it is a cost-effective reliability upgrade. 🔋
The best Linux gaming distro in 2026 depends on hardware - SteamOS Holo powers about 26% of Linux Steam installs but targets AMD, while Bazzite, Arch, and Mint lead for broader hardware. Bazzite gives the closest console-like experience for custom desktops.
How to fix low FPS after a USB DAC upgrade starts with audio drivers, USB bandwidth, and sample-rate checks. Re-test games with other ports, exclusive mode off, and background audio tools closed before blaming graphics hardware.
The real world difference between PCIe 5.0 and PCIe 4.0 for MMO and live-service games is usually about loading, patches, and headroom. SA players should weigh NVMe speed, GPU lanes, CPU limits, and platform cost.
8000Hz polling is overrated for most SA gamers because it can add system load without a clear feel upgrade. Check FPS consistency, mouse sensor quality, USB stability, and the games you play before chasing the highest rate.
B850 vs B650 depends on Competitive esports needs, platform limits, and budget fit. Compare compatibility, latency, upgrade path, and real setup constraints before choosing for an SA build.
