LLM Providers & Models

Mudabbir supports six LLM providers, from free local models to cloud APIs. This guide covers everything: which provider to pick, which models to use, how they interact with agent backends, and how to configure them.

Quick Start: Which Provider Should I Use?

Provider Details

Anthropic (Default)

The recommended provider for maximum capability. Claude models excel at coding, tool use, and complex reasoning.

Configuration:

export MUDABBIR_LLM_PROVIDER="anthropic"
export MUDABBIR_ANTHROPIC_API_KEY="sk-ant-..."
export MUDABBIR_ANTHROPIC_MODEL="claude-sonnet-4-5-20250929"

Available Models:

Gemini

Google’s Gemini models via their OpenAI-compatible API. Free tier available with generous limits. Internally, Mudabbir routes Gemini through the OpenAI-compatible code path using Google’s endpoint at https://generativelanguage.googleapis.com/v1beta/openai/.

Configuration:

export MUDABBIR_LLM_PROVIDER="gemini"
export MUDABBIR_GOOGLE_API_KEY="AIza..."
export MUDABBIR_GEMINI_MODEL="gemini-2.5-flash"

Available Models:

Ollama (Local)

Run models entirely on your machine. No API keys, no cloud, no costs. Requires Ollama installed and running.

Configuration:

export MUDABBIR_LLM_PROVIDER="ollama"
export MUDABBIR_OLLAMA_HOST="http://localhost:11434"
export MUDABBIR_OLLAMA_MODEL="llama3.2"

Recommended Models:

Verify connectivity:

mudabbir --check-ollama

This runs 4 checks: server reachable, model available, API compatibility, and tool calling support.

OpenAI

Use OpenAI’s GPT models directly.

Configuration:

export MUDABBIR_LLM_PROVIDER="openai"
export MUDABBIR_OPENAI_API_KEY="sk-..."
export MUDABBIR_OPENAI_MODEL="gpt-4o"

Available Models:

OpenAI-Compatible

Connect to any endpoint that implements the OpenAI Chat Completions API. This includes hosted services (OpenRouter, Together AI, Fireworks) and self-hosted servers (vLLM, LiteLLM, text-generation-inference).

Configuration:

export MUDABBIR_LLM_PROVIDER="openai_compatible"
export MUDABBIR_OPENAI_COMPATIBLE_BASE_URL="https://openrouter.ai/api/v1"
export MUDABBIR_OPENAI_COMPATIBLE_API_KEY="sk-or-..."
export MUDABBIR_OPENAI_COMPATIBLE_MODEL="anthropic/claude-3.5-sonnet"
export MUDABBIR_OPENAI_COMPATIBLE_MAX_TOKENS=0  # 0 = no limit

Popular services:

Verify connectivity:

mudabbir --check-openai-compatible

This runs 2 checks: API connectivity and tool calling support.

Auto (Default)

When llm_provider is set to "auto" (the default), Mudabbir auto-detects the best available provider:

1. Anthropic — if MUDABBIR_ANTHROPIC_API_KEY is set
2. OpenAI — if MUDABBIR_OPENAI_API_KEY is set
3. Ollama — fallback (no key needed)

This means Mudabbir works out of the box with Ollama if no API keys are configured.

Backend Compatibility Matrix

Not every provider works with every agent backend. Here’s the full compatibility matrix:

Claude Agent SDK (Recommended)

The default and most capable backend. Uses the official Claude Agent SDK as a subprocess with built-in agentic tools.

Important: API format requirement. The Claude SDK uses the Anthropic Messages API format for all communication. This means it only works with endpoints that understand Anthropic’s message format — not endpoints that only speak OpenAI’s Chat Completions format.

How it connects to providers:

All providers are passed to the Claude CLI subprocess via environment variables (ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY):

Built-in SDK tools: Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch — these are provided by the Claude SDK itself and don’t need configuration.

Fast-path optimization: For messages classified as SIMPLE by the model router, Mudabbir bypasses the CLI subprocess entirely and calls the Anthropic API directly. This saves ~1.5–3 seconds of subprocess startup time per simple message. The fast-path requires an API key.

Security: Uses PreToolUse hooks to intercept and block dangerous Bash commands before execution.

Persistent client: The subprocess is reused across messages. It only reconnects when the model or tool configuration changes, avoiding repeated startup overhead.

MCP servers: Fully supported. Loads enabled MCP server configs and passes them to the SDK subprocess. Supports stdio, SSE, and HTTP transports.

Best for: Anthropic and Ollama users who want maximum capability, coding tasks, complex multi-step reasoning, and tool-heavy workflows.

Mudabbir Native

Custom orchestrator that uses the Anthropic SDK for reasoning and Open Interpreter for code execution. Provides a transparent, user-controlled agentic loop with integrated security layers. This is the most versatile backend — it supports every provider by auto-converting between API formats.

How it connects to providers:

Mudabbir Native creates different API clients depending on the provider, automatically handling format differences:

Automatic format conversion: When using Gemini or OpenAI-Compatible providers, Mudabbir Native automatically:

• Converts Anthropic-format tool definitions to OpenAI function-calling format
• Converts message history between Anthropic and OpenAI formats
• Maps OpenAI’s finish_reason to Anthropic’s stop_reason
• Handles tool_calls response objects with id, function.name, function.arguments

This means you get the same tool-use experience regardless of provider — the conversion is invisible.

API timeouts:

Built-in tools: shell, read_file, write_file, edit_file, list_dir, remember, recall, forget — plus any MCP tools configured.

Security layers:

1. Dangerous command regex — blocks rm -rf /, mkfs, dd, etc.
2. Sensitive path protection — SSH keys, AWS credentials, .env files
3. File jail — restricts filesystem access to ~/.mudabbir/ by default
4. Output redaction — hides API keys and passwords from responses

Smart routing: Supported for Anthropic provider only. Disabled for Ollama and Gemini.

MCP servers: Supported. Tools are registered with mcp_<server>__<tool> naming convention.

Best for: Users who want fine-grained control, transparency, and strong security guardrails. Works well with Gemini for a free, full-featured setup.

Open Interpreter

Lightweight wrapper around Open Interpreter. Delegates model management entirely to Open Interpreter’s own provider system, which uses LiteLLM under the hood.

How it connects to providers:

Ollama prefix: Ollama models are automatically prefixed with ollama/ (e.g., llama3.2 becomes ollama/llama3.2). This is required by LiteLLM to identify the provider.

Streaming: Runs in a thread pool with queue-based chunk streaming. Filters verbose console output and emits only user-facing messages and tool results.

Tool execution: Open Interpreter handles code execution directly — it generates and runs code in a sandboxed environment. Mudabbir wraps the output into standardized tool_use/tool_result events for the Activity panel.

Limitations:

• No MCP server support
• No smart model routing
• No PreToolUse security hooks
• Less granular tool control compared to other backends

Best for: Quick prototyping, users familiar with Open Interpreter, scenarios where code execution is the primary use case.

Smart Model Routing

When using Anthropic as the provider, Mudabbir can automatically select the model size based on message complexity. This optimizes cost by using cheaper models for simple queries.

How It Works

Each incoming message is classified into a tier:

Classification Signals

• Simple patterns: “hi”, “hello”, “thanks”, “what is X?”, “remind me”, short messages under 30 characters
• Complex signals: “plan”, “architect”, “debug”, “implement”, “analyze”, “optimize”, “research”
• Threshold: 1 complex signal + message > 30 chars = COMPLEX; 2+ complex signals = COMPLEX; > 400 chars = COMPLEX

Configuration

export MUDABBIR_SMART_ROUTING_ENABLED=true
export MUDABBIR_MODEL_TIER_SIMPLE="claude-haiku-4-5-20251001"
export MUDABBIR_MODEL_TIER_MODERATE="claude-sonnet-4-5-20250929"
export MUDABBIR_MODEL_TIER_COMPLEX="claude-opus-4-6"

Configuration Reference

All LLM-Related Settings

Setting via config.json

All settings can also be set in ~/.mudabbir/config.json:

{
  "llm_provider": "gemini",
  "gemini_model": "gemini-2.5-flash",
  "smart_routing_enabled": false
}

Setting via Dashboard

Open the web dashboard (default at http://localhost:8888) and go to Settings > General to select your provider and model from dropdown menus.

Security Notes

• API keys are encrypted at rest using Fernet + PBKDF2 in ~/.mudabbir/secrets.enc. They are never stored in plaintext in config.json.
• Environment variables override config file values and are never written to disk.
• The Gemini provider reuses the google_api_key field, which is also used for image generation. One key covers both features.

Troubleshooting

Ollama: “Model not found”

ollama list              # See available models
ollama pull llama3.2     # Download a model
mudabbir --check-ollama # Run connectivity check

Gemini: “API key invalid”

1. Go to AI Studio and create/verify your key
2. Make sure the key is saved in Settings > API Keys > Google API Key
3. Ensure the Gemini API is enabled for your Google Cloud project

OpenAI-Compatible: “Cannot connect”

mudabbir --check-openai-compatible  # Run connectivity check

Verify:

• The base URL is correct and includes /v1 if needed
• The server is running and accessible
• The model name matches what the endpoint expects

Wrong provider being used

If llm_provider is "auto", Mudabbir picks the first available key (Anthropic > OpenAI > Ollama). Set an explicit provider to override:

export MUDABBIR_LLM_PROVIDER="gemini"