Ollama (Local LLMs)

Run Mudabbir entirely on your machine with Ollama — no API keys, no cloud, no costs. Both the Claude Agent SDK and Mudabbir Native backends work with Ollama out of the box.

How It Works

Since Ollama v0.14.0, Ollama exposes an Anthropic Messages API compatible endpoint. Mudabbir points the same AsyncAnthropic client (or Claude SDK subprocess) at your local Ollama server instead of Anthropic’s cloud. Same tool format, same streaming, zero format conversion.

Prerequisites

curl -fsSL https://ollama.com/install.sh | sh

ollama pull qwen2.5:7b       # Good balance of speed and quality
# or
ollama pull llama3.2          # Default model

ollama serve

Quick Start

export MUDABBIR_LLM_PROVIDER=ollama
export MUDABBIR_OLLAMA_MODEL=qwen2.5:7b
mudabbir

{
  "llm_provider": "ollama",
  "ollama_host": "http://localhost:11434",
  "ollama_model": "qwen2.5:7b"
}

Verify Setup

Run the built-in connectivity check:

mudabbir --check-ollama

This performs 4 checks:

Configuration

Auto-Detection

When llm_provider is "auto" (the default):

1. If anthropic_api_key is set → uses Anthropic
2. If no API key is set → falls back to Ollama automatically

This means if you install Mudabbir and Ollama without any API keys, it just works.

Compatible Backends

Claude Agent SDK + Ollama

The default backend. The SDK subprocess receives these environment variables:

• ANTHROPIC_BASE_URL → your Ollama host
• ANTHROPIC_API_KEY → "ollama" (accepted but not validated)

All SDK built-in tools (Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch) work as usual.

Mudabbir Native + Ollama

The custom orchestrator creates AsyncAnthropic(base_url=ollama_host) and sends tool definitions in standard Anthropic format. The model receives tool schemas and returns tool_use blocks just like Claude.

Recommended Models

Limitations

• Smart Model Router is skipped — When using Ollama, the Model Router cannot switch between models. Smart routing is automatically disabled.
• Tool calling quality varies — Smaller models may not use tools reliably. If tools aren’t being called, try a larger model.
• Ollama v0.14.0+ required — Older versions don’t expose the Anthropic Messages API endpoint.

Error Messages

Mudabbir provides Ollama-specific error messages instead of generic API errors:

Troubleshooting

”Model not found in Ollama”

This means Mudabbir is trying to use a model you haven’t pulled. The default model is llama3.2 — if you use a different model (e.g., deepseek), make sure to update the setting:

# Check what models you have
ollama list

# Set the correct model
export MUDABBIR_OLLAMA_MODEL="deepseek-r1:8b"

Or update it in the dashboard: Settings → General → Ollama Model.

”Cannot reach Ollama server"

# Check Ollama is running
ollama serve

# Verify it's listening
curl http://localhost:11434/api/tags

"Messages API failed”

Your Ollama version may be too old. Update:

# macOS
brew upgrade ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

“Model responded but did not use the tool”

Try a more capable model:

ollama pull qwen2.5:14b

Then set ollama_model to qwen2.5:14b in Settings or config.

Implementation