How to Run Your First Local Model With Ollama

Running an AI model on your own machine sounds like it should need a CUDA degree and a weekend of config files. It does not. Ollama collapses the whole process into one terminal command that downloads a model and drops you into a live chat session, with no internet connection required once the weights are on disk. If you can open a terminal, you can have a private chatbot answering questions on your own PC in about fifteen minutes.

Quick Answer

Type ollama run llama3.2 in your terminal. Ollama downloads the model weights on first use, around 2 GB for the compact 3B model, then opens an interactive prompt where you type a question and press enter. That single command is your entire first run.

Step 1: Install Ollama

Download the installer for your operating system from the official Ollama site and run it. On Windows and macOS it installs as a background application that quietly starts a local server; on Linux a one-line install script does the same. Once it is running, the engine listens locally and waits for commands.

You can confirm the install worked by opening a terminal and typing ollama --version. If it prints a version number, you are ready. Nothing has been downloaded yet, because Ollama only fetches a model when you first ask for one.

A capable everyday PC handles small models comfortably. If you are building or upgrading specifically for local AI work, the AI-ready PCs at Evetech pair a modern GPU with enough memory to run larger models without the long waits.

Step 2: Pull and run your first model

This is the moment everything happens. In your terminal, type:

ollama run llama3.2

On first use Ollama recognises it does not have the model locally, so it pulls the weights, roughly 2 GB for the 3B version, and shows a download bar. When that finishes it launches an interactive session and you see a prompt waiting for input. Type a question, press enter, and the model answers entirely on your hardware.

Every time you run that command afterwards, launch is instant because the weights are already cached. To leave the chat, type /bye or press Ctrl and D.

Step 3: Choose the right model for your hardware

The model you pick should match your RAM and patience. Three sensible starting points in 2026:

• Llama 3.2 (3B) is the lightest, around 2 GB, and runs on almost any machine. Best for a first test.
• Gemma 3 (4B) from Google is around 3.3 GB and punches above its weight on instruction following.
• Llama 3.3 (8B) is the balanced all-rounder at roughly 4.9 GB, wanting about 8 GB of RAM, and gives noticeably stronger answers.

Bigger models give better responses but need more memory and run slower without a capable GPU. Start small, then size up once you know what your machine handles. Browsing the best-selling PCs at Evetech is a fast way to gauge what current builds offer in memory and graphics headroom.

Step 4: The handful of commands worth knowing

You do not need many. These five cover almost everything a beginner does:

• ollama run downloads if needed and starts a chat.
• ollama pull downloads without launching, useful for prepping ahead.
• ollama list shows every model on your machine.
• ollama rm deletes a model to free disk space.
• ollama serve starts the API server so other apps can talk to your models.

That last one matters once you move past chatting in the terminal. The local server lets desktop front-ends and your own scripts send prompts to the model, which is how most people graduate from a toy to a genuinely useful local assistant.

Why bother running models locally?

Three reasons. Privacy, because nothing you type leaves your machine, which suits sensitive work or just keeping personal notes off someone else's servers. Cost, because there are no per-token API fees no matter how much you use it. And availability, because once the weights are downloaded you can work fully offline.

The trade-off is hardware. A cloud model runs on a data-centre GPU; yours runs on what is in your case. That is exactly why model choice and a decent GPU matter, and why starting with a small model on the machine you already own is the smart first move.

Frequently Asked Questions

Do I need a powerful GPU to use Ollama?

No, not to start. A small model like Llama 3.2 3B runs on ordinary hardware, just more slowly without a dedicated GPU. A modern graphics card and 16 GB or more of RAM make larger models practical and far quicker.

How much disk space do models take?

Small models are around 2 to 3 GB, mid-size ones 5 GB or so, and large ones can reach 16 GB or more. Each model you pull is stored locally until you remove it with ollama rm, so keep an eye on free space.

Does Ollama work offline?

Yes. You need an internet connection only for the first download of each model. After the weights are on disk, you can run that model with no connection at all, which is one of the main attractions.

How do I exit a chat session?

Type /bye at the prompt or press Ctrl and D. The model stays downloaded, so running the same command again starts instantly without re-fetching anything.

Which model should a complete beginner start with?

Start with Llama 3.2 3B. It is small, fast and forgiving of modest hardware, so you learn the workflow without long downloads or sluggish replies. Move up to an 8B model once you know your machine can handle it.