Skip to main content

Running Locally

Dependencies​

  • ClickHouse (for request tracing & analytics)
  • PostgreSQL (for metadata and user management)
  • Redis (for caching and rate‑limiting)

Launch Options​

You can self host our enterprise version using two options.

Using binary​

Supported Platforms:

  • x86_64
  • aarch64
ai-gateway-enteprise serve -c ai-gateway.yaml

Using docker​

docker run -it 
    -p 8080:8080 \
    <private-url>/ai-gateway-enteprise serve \
    -c ai-gateway.yaml

Make Your First Request​

Test the gateway with a simple chat completion:

# Chat completion with GPT-4
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

# Or try Claude
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-opus",
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

Using MCPs Servers​

Invoke an MCP server alongside your request:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Ping the server using the tool and return the response"}],
    "mcp_servers": [{"server_url": "http://localhost:3004"}]
  }'

Next Steps:​

Refer to https://docs.langdb.ai/ for understanding various features.