Ollama Local Models

📘 Guide Status: This page covers configuring Ollama as an OpenClaw model provider for running local LLMs (Llama, Mistral, CodeLlama, and more) with no API key and full data sovereignty. For first-time OpenClaw setup, see the Quick Start Guide. For the full config layout, see Configuration File Structure.

Overview

Ollama is an open-source tool that runs large language models (LLMs) locally on your Mac, Linux, or Windows machine. It provides a simple API (typically at http://localhost:11434) and supports many models-Meta’s Llama, Mistral, CodeLlama, Phi, Gemma, and others-with no cloud account or API key. OpenClaw is model-agnostic: you can use Ollama as the “brain” of your agent so that all prompts and responses stay on your hardware. That means full privacy, no per-token cost after you have the hardware, and no dependency on external APIs-ideal for sensitive data, air-gapped setups, or high-volume use where cloud API cost would add up.

This guide covers:

  • What Ollama is and why use it with OpenClaw
  • Prerequisites: OpenClaw installed, Ollama installed and running, hardware considerations
  • Installing Ollama and pulling a model
  • Adding the Ollama provider block to your OpenClaw config (endpoint, model names)
  • Choosing models (Llama, Mistral, CodeLlama, etc.) and assigning them to agents
  • Verification, common issues, and best practices
  • Links to agents, memory, and other model providers

Prerequisites

  • OpenClaw installed and running (installation guide, platform-specific installs)
  • Ollama installed on the same machine (or a reachable host) and at least one model pulled (e.g. ollama pull llama3.2)
  • Sufficient hardware: Ollama runs on CPU but benefits from GPU (e.g. NVIDIA, Apple Silicon, or AMD). RAM and disk depend on model size (e.g. 8GB+ RAM for smaller models; 16GB+ for larger ones). See ollama.com for requirements.
  • Basic familiarity with the command line and editing config (e.g. openclaw config edit)

Step 1: Install Ollama and Pull a Model

Ollama must be installed and running before OpenClaw can use it. No API key is required.

  1. Install Ollama from ollama.com (download for macOS, Linux, or Windows). On Linux you can also use the install script from the official site. On macOS and Windows, Ollama often runs as a background service after installation.
  2. Confirm Ollama is running. Open a terminal and run: 
    ollama list
    If the command works, the Ollama service is up. If not, start it (e.g. launch the Ollama app or run ollama serve in the background).
  3. Pull at least one model. For example: 
    ollama pull llama3.2
ollama pull mistral
    Use the exact model name (e.g. llama3.2, mistral, codellama) in your OpenClaw config later. Model names are case-sensitive.

The default API endpoint is http://localhost:11434. If you run Ollama on another host or port, you will set that in the OpenClaw provider config.

Step 2: Add the Ollama Provider in Config

Edit your OpenClaw config with openclaw config edit (or open the file path shown by openclaw doctor). In the model providers section, add a block for Ollama. Exact keys depend on your OpenClaw version; typical pattern:

  • Provider id: e.g. ollama (you’ll reference this in your agent config).
  • Base URL / endpoint: Usually http://localhost:11434. If Ollama runs on another machine or port, set the full base URL (e.g. http://192.168.1.10:11434). For security, avoid exposing Ollama to the public internet; keep it on localhost or a trusted network. See security best practices.
  • Models: List of model names you have pulled in Ollama (e.g. llama3.2, mistral, codellama). Only these names will be allowed for agents using this provider.
  • No API key: Ollama does not use API keys; you do not need to set any key in the config or environment for Ollama itself.
  • Optional: Request timeout or other limits if your OpenClaw version supports them.

After editing, save the file. Restart the gateway if required by your version so it picks up the new provider. For the full config layout (gateway, channels, model providers, agents, skills, memory), see Configuration File Structure.

Step 3: Choose an Ollama Model

Ollama supports many models. Use the exact tag name returned by ollama list (e.g. llama3.2, mistral, codellama). Examples of popular models and typical use:

Model (example tag) Typical use
llama3.2, llama3.1 General chat, reasoning, and instruction-following; good balance of size and quality.
mistral Efficient and fast; good for quick Q&A and high-volume, lower-resource use.
codellama Code generation and explanation; useful for development workflows with OpenClaw.
phi3, phi4 Smaller models; lower RAM/GPU requirements, suitable for lightweight or edge use.
gemma2, qwen2 Alternative general-purpose and multilingual models available via Ollama.

Match the model’s context length to your memory and agent context settings in OpenClaw so you don’t exceed the model’s limit. Larger models need more RAM or GPU; if you see out-of-memory errors, try a smaller model or reduce context.

Step 4: Assign the Model to an Agent

In the agents section of your config, set the agent’s model to the Ollama provider and the desired model name (e.g. llama3.2 or mistral). Only model names listed under your Ollama provider block will be allowed. You can use Ollama for one agent and a cloud provider (e.g. Claude) for another-see Agent customization for multiple agents and Model Providers overview for mixing cloud and local.

Verification

Run openclaw status and ensure the gateway and config are valid. Confirm Ollama is running (ollama list) and that the endpoint (e.g. http://localhost:11434) is reachable from the machine where OpenClaw runs. Send a test message to your agent on a connected channel (e.g. Telegram). If the agent responds using the local model, the provider is working. For diagnostics, use openclaw doctor and openclaw logs; see the troubleshooting guide and model and API errors for common failures.

Common Issues (Ollama)

Issue Solution
Connection refused / Ollama not responding Ensure Ollama is running (ollama list or start the Ollama app). If OpenClaw runs in Docker or on another host, confirm the base URL (e.g. http://host:11434) is correct and reachable (firewall, network).
Model not found / invalid model Pull the model with ollama pull <modelname> and use the exact tag from ollama list in your OpenClaw config. Model names are case-sensitive.
Timeout or very slow responses Local models can be slower than cloud APIs, especially on CPU. Use a smaller or quantized model, or ensure GPU is available for Ollama. Increase timeout in provider config if supported.
Out of memory (OOM) Choose a smaller model or one with lower RAM requirements (e.g. phi3, or a smaller quantized variant). Close other applications or run Ollama on a machine with more RAM/VRAM.
Wrong or outdated base URL Default is http://localhost:11434. If you changed Ollama’s port or host, update the provider’s base URL in OpenClaw config and restart the gateway.

For more errors and fixes, see the full troubleshooting hub and model and API errors.

Best Practices

  • Endpoint security: Keep Ollama bound to localhost or a trusted private network. Do not expose port 11434 to the internet. If OpenClaw and Ollama run on different machines, use a VPN or private network. See network isolation and security best practices.
  • Hardware: Use a GPU (NVIDIA, Apple Silicon, or AMD) when possible for faster inference. Match model size to your RAM/VRAM to avoid OOM.
  • Context and memory: Align your memory and agent context with the model’s context window to avoid truncation or errors.
  • Updates: Keep Ollama and your models updated (ollama pull <model> to refresh). Keep OpenClaw and config in sync with the official docs.
  • Hybrid setup: Use Ollama for privacy-sensitive or high-volume tasks and a cloud provider (e.g. Claude or OpenAI) for complex reasoning when needed; configure multiple providers and assign models per agent.

Other Model Providers

You can use multiple providers: e.g. Ollama for local, private tasks and Claude or OpenAI for complex ones. See the Model Providers overview and the per-provider guides: Anthropic Claude, OpenAI GPT, Google Gemini, DeepSeek, Local Models (Ollama).

Next Steps

Related Resources