Luthien

Try the Luthien Proxy Alpha

The Luthien Proxy is now available in alpha. It's a Claude Code and OpenAI Codex CLI compatible LLM proxy that brings Redwood-style AI Control to production agentic deployments.

View on GitHub →

Prerequisites: Docker, Python 3.13+, and uv package manager

Get started in minutes:

# Clone the repository
git clone https://github.com/luthienresearch/luthien-proxy.git
cd luthien-proxy

# Configure environment
cp .env.example .env
# Edit .env and add your API keys:
# - OPENAI_API_KEY
# - ANTHROPIC_API_KEY

# Start the proxy
./scripts/quick_start.sh

# Test it works
./scripts/test_v2_gateway.sh

# Launch Claude Code or Codex with the proxy
./scripts/launch_claude_code.sh
# or
./scripts/launch_codex.sh

Verify Everything is Working

Once the proxy is running, navigate to http://localhost:8000/v2/activity/monitor to see the activity monitor. This real-time dashboard shows all requests flowing through the proxy, policy decisions, and detailed event streams. As you interact with Claude Code or Codex, you'll see requests, policy events, and responses appear in the monitor.

The proxy provides policy orchestration, decision logic, and full observability for your AI systems. Learn more about our approach.

Custom Policies

The proxy gives you complete control over requests and responses in-flight. Policies let you monitor, modify, block, or enhance any aspect of LLM interactions. For most use cases, we recommend using SimpleEventBasedPolicy, which provides a straightforward hook-based interface.

Configure an Existing Policy

Start by trying one of the built-in policies. Edit config/v2_config.yaml to enable the SimpleToolFilterPolicy that blocks dangerous tools:

policy:
  class: "luthien_proxy.v2.policies.simple_tool_filter_example:SimpleToolFilterPolicy"
  config:
    blocked_tools:
      - "execute_code"
      - "Bash"

Restart the proxy and watch the activity monitor at localhost:8000/v2/activity/monitor to see policy events as requests flow through. Try asking your LLM to run bash commands and watch them get blocked!

Implement Your Own Policy

Create your policy class in src/luthien_proxy/v2/policies/my_policy.py:

from luthien_proxy.v2.policies.simple_event_based_policy import SimpleEventBasedPolicy
from luthien_proxy.v2.policies.policy_context import PolicyContext

class MyPolicy(SimpleEventBasedPolicy):
    def __init__(self, config):
        super().__init__(config)
        self.my_setting = config.get("my_setting", "default")

    async def on_request(self, request, context: PolicyContext):
        # Inspect or modify requests before they reach the LLM
        context.emit("policy.request", f"Processing request with {self.my_setting}")
        return request

    async def on_response_content(self, content, context: PolicyContext, streaming_ctx):
        # Called when complete content blocks finish
        context.emit("policy.content", f"Received {len(content)} characters")

        # Example: transform content
        if "dangerous" in content.lower():
            return "[Content filtered for safety]"

        return content  # Return unchanged content

    async def on_response_tool_call(self, block, context: PolicyContext, streaming_ctx):
        # Called when a complete tool call finishes buffering
        tool_name = block.name
        tool_args = block.arguments
        context.emit("policy.tool_complete", f"Tool call: {tool_name}")

        # Example: block dangerous operations
        if tool_name == "Bash" and "rm -rf" in tool_args:
            context.emit("policy.blocked", "Blocked dangerous command")
            return None  # Block this tool call

        return block  # Allow the tool call

Key concepts:

SimpleEventBasedPolicy automatically handles streaming and buffering
on_request() lets you inspect or modify requests before they reach the LLM
on_response_content() gives you complete content blocks as strings
on_response_tool_call() gives you complete tool calls with full arguments
Return None from on_response_tool_call() to block a tool call
Return modified content/blocks to transform responses
context.emit() sends events visible in the activity monitor
context.scratchpad stores state that persists across hooks for a single request

Then configure it in config/v2_config.yaml:

policy:
  class: "luthien_proxy.v2.policies.my_policy:MyPolicy"
  config:
    my_setting: "custom_value"

See simple_uppercase_example.py for a simple content transformation example, or simple_tool_filter_example.py for tool call blocking.

Advanced: AI-Supervised Tool Calls

For production deployments requiring stronger safety guarantees, the tool_call_judge_v3.py policy demonstrates advanced streaming capabilities and local LLM review. It intercepts tool calls as they arrive, sends them to a local language model for safety evaluation, and blocks dangerous operations before they execute. This approach combines the speed of streaming with the intelligence of AI-powered decision-making, enabling real-time protection without sacrificing responsiveness.

Early Development