> For the complete documentation index, see [llms.txt](/llms.txt).
> Markdown versions of each page are available by appending .md to any URL.

# Set up Ollama

Install Ollama, run LLMs locally, compare model performance, and integrate local models into your apps using Warp.

Run AI models locally — with no API costs, no data leaving your machine, and your choice of model. This guide walks through installing Ollama, running your first model, integrating it into an existing app, and customizing model behavior using Warp.

![Setting up Ollama in Warp video](https://i.ytimg.com/vi/Aq8vDxUg4VE/sddefault.jpg)

## Prerequisites

**Hardware requirements**

You'll need enough memory to load the model weights. As a general estimate, plan on roughly 1GB of memory per billion parameters.

| Model size | Example models | Minimum RAM/VRAM |
| --- | --- | --- |
| 3B | Llama 3.2 3B, Phi-3 Mini (3.8B) | 4GB |
| 7–8B | Mistral 7B, Llama 3.1 8B | 8GB |
| 13B | Llama 2 13B | 16GB |
| 22–34B | Codestral (22B), Mixtral (46.7B total) | 24GB |
| 70B | Llama 3.1 70B | 48GB |

On Apple Silicon Macs, unified memory is shared between CPU and GPU. An M2 Mac with 16GB unified memory can run 7–8B models comfortably.

**Software requirements**

-   macOS 14 Sonoma+, Windows 10+, or Ubuntu 20.04+
-   No account required — Ollama is free and open source

## 1\. Install Ollama

**macOS**

Run the install script in your terminal:

```
curl -fsSL https://ollama.com/install.sh | sh
```

Or download the macOS app manually from [ollama.com/download](https://ollama.com/download/Ollama.dmg). The app runs as a background service and appears in your menu bar.

**Linux**

```
curl -fsSL https://ollama.com/install.sh | sh
```

On systemd-based Linux distributions, the installer typically registers Ollama as a service. To start it and enable it on boot, run:

```
sudo systemctl enable ollamasudo systemctl start ollama
```

**Windows**

Run this in PowerShell:

```
irm https://ollama.com/install.ps1 | iex
```

Or download the installer manually from [ollama.com/download](https://ollama.com/download). Ollama runs as a background service after installation.

## 2\. Run your first model

Pull a model to download it locally. This example uses Llama 3.2 (3B), a lightweight model that runs on most hardware:

```
ollama pull llama3.2
```

Then start an interactive chat session:

```
ollama run llama3.2
```

Type your prompt and press Enter. Type `/bye` to exit the session.

To try a larger model with stronger reasoning:

```
ollama run llama3.1:8b
```

Use `ollama ls` to see all models you've downloaded, and `ollama ps` to see which models are currently loaded in memory.

## 3\. Understand model types

When browsing [ollama.com/library](https://ollama.com/library), you'll see labels on models that indicate their capabilities:

| Label | Meaning |
| --- | --- |
| **thinking** | The model reasons step-by-step before answering; better for complex problems. |
| **tools** | The model can call external functions or utilities (e.g., web search). |
| **vision** | The model can process and respond to images. |
| **embedding** | Converts text to numeric vectors for search or RAG pipelines. |

Quantized models (e.g., `q4_0`, `q8_0` variants) use significantly less memory than full-precision originals. Start with a quantized version if you're memory-constrained.

## 4\. Compare model performance

To benchmark models side by side, Warp makes it easy to run prompts in parallel tabs and compare results. Pass a prompt and the `--verbose` flag to get timing stats alongside the response:

```
ollama run --verbose llama3.2 "Summarize the Rust ownership model in one paragraph"
```

Run the same prompt on two different models to compare output quality and throughput before committing to one.

To see which models are currently loaded and their memory usage:

```
ollama ps
```

## 5\. Integrate Ollama into your app

Ollama exposes an OpenAI-compatible REST API at `http://localhost:11434/v1/`. You can drop it in wherever you're currently using the OpenAI API — just update three values:

1.  **Base URL** → `http://localhost:11434/v1/`
2.  **API key** → any non-empty string (e.g., `"ollama"` — required by the SDK but ignored by Ollama)
3.  **Model name** → the name of your local model (e.g., `"llama3.2"`)

**Python (OpenAI SDK)**

```
from openai import OpenAI
client = OpenAI(    base_url="http://localhost:11434/v1/",    api_key="ollama",  # required by the SDK but ignored by Ollama)
response = client.chat.completions.create(    model="llama3.2",    messages=[        {"role": "user", "content": "Explain async/await in Python"}    ])
print(response.choices[0].message.content)
```

**Node.js (OpenAI SDK)**

```
import OpenAI from "openai";
const client = new OpenAI({  baseURL: "http://localhost:11434/v1/",  apiKey: "ollama", // required by the SDK but ignored by Ollama});
const response = await client.chat.completions.create({  model: "llama3.2",  messages: [{ role: "user", content: "Explain async/await in Python" }],});
console.log(response.choices[0].message.content);
```

Ollama also provides official native libraries if you prefer an SDK built specifically for Ollama rather than the OpenAI compatibility layer: `pip install ollama` for Python and `npm i ollama` for JavaScript. See the [Ollama README](https://github.com/ollama/ollama) for details.

Open your app's code in Warp and use the agent to locate the OpenAI client initialization, swap in the new values, and test the connection — all from the same terminal session.

## 6\. Customize model behavior

You can create a custom model variant by writing a `Modelfile` — a configuration file that sets a system prompt, adjusts generation parameters, or builds on any existing Ollama model.

Create a file called `Modelfile`:

```
FROM llama3.2
SYSTEM """You are a focused code review assistant.Always identify security issues first, then logic errors, then style.Keep feedback concise and actionable."""
PARAMETER temperature 0.3PARAMETER num_ctx 8192
```

Then build and run your custom model:

```
ollama create code-reviewer -f ./Modelfileollama run code-reviewer
```

Your custom model is saved locally and available any time you run `ollama run code-reviewer`. Ask Warp's agent to generate a Modelfile for a specific use case — describe what you want the model to do, and let the agent write the configuration.

## Productivity tips

-   **Unload to free memory** — Use `ollama stop <model>` to unload a model from memory before loading a larger one.
-   **Script with the REST API** — For quick testing without the SDK: `curl http://localhost:11434/api/chat -d '{"model": "llama3.2", "messages": [{"role": "user", "content": "Hello"}], "stream": false}'`
-   **Keep a model inventory** — `ollama ls` shows all locally cached models with their sizes and download dates. Use `ollama rm <model>` to delete models you no longer need.
-   **Compare models in parallel** — Open side-by-side Warp tabs and run the same prompt against two different models to compare quality and speed before deciding which to use in your app.

## Next steps

You have Ollama installed, a model running locally, and a path to integrate it into any app using the OpenAI-compatible API.

Explore related guides and features:

-   [Set up Claude Code](/guides/external-tools/how-to-set-up-claude-code/) or [Set up Codex CLI](/guides/external-tools/how-to-set-up-codex-cli/) — use cloud-based agents alongside your local Ollama setup
-   [Run multiple agents at once](/guides/agent-workflows/how-to-run-multiple-ai-coding-agents/) — run Ollama and a cloud agent in parallel to compare outputs on the same task
-   [Ollama model library](https://ollama.com/library) — browse all available models with size and capability details
-   [Ollama documentation](https://github.com/ollama/ollama/blob/main/README.md) — advanced configuration, GPU setup, and environment variables