Example: Routing with Interceptors and Compliance

This example showcases a sophisticated routing configuration that uses pre-request interceptors to enforce usage quotas and guardrails, while handling region-specific compliance and prioritizing performance for premium users.

Goals:

1. Enforce a daily rate limit on all users to prevent abuse.
2. Check all requests for policy violations using a semantic guardrail.
3. Provide high-performance models for premium users in the EU, but only if they are reliable.
4. Ensure GDPR compliance by using a specialized model for requests with that requirement.
5. Provide a clear error message to users who have exceeded their quota.

Complete Chat Completion Request with Interceptors and Compliance:

{
  "model": "router/dynamic",
  "messages": [
    {
      "role": "system",
      "content": "You are a financial advisory assistant that provides investment recommendations and portfolio analysis for premium clients in the European market."
    },
    {
      "role": "user",
      "content": "I need urgent advice on my cryptocurrency portfolio diversification strategy. Given the current market volatility, should I reallocate my €2M investment across different digital assets? Please provide detailed analysis including tax implications under GDPR compliance requirements."
    }
  ],
  "stream": true,
  "extra": {
    "user": {
      "tier": "premium"
    }
  },
  "router": {
    "type": "conditional",
    "pre_request": [
      {
        "name": "rate_limiter",
        "type": "interceptor",
        "limit": 1000,
        "period": "day",
        "target": "user_id"
      },
      {
        "name": "semantic_guardrail",
        "type": "guardrail"
      }
    ],
    "routes": [
      {
        "name": "premium_eu_high_performance",
        "conditions": {
          "all": [
            {
              "extra.user.tier": {
                "$eq": "premium"
              }
            },
            {
              "metadata.region": {
                "$eq": "EU"
              }
            },
            {
              "pre_request.rate_limiter.passed": {
                "$eq": true
              }
            }
          ]
        },
        "targets": {
          "$any": [
            "anthropic/claude-4-opus",
            "openai/gpt-o3",
            "gemini/gemini-2.5-pro"
          ],
          "filter": {
            "error_rate": {
              "$lt": 0.02
            }
          },
          "sort_by": "ttft",
          "sort_order": "min"
        }
      },
      {
        "name": "gdpr_compliance_fallback",
        "conditions": {
          "metadata.tags.gdpr": {
            "$eq": "true"
          }
        },
        "targets": "eu-specialist/gdpr-compliant-model"
      },
      {
        "name": "semantic_guardrail_violation_block",
        "conditions": {
          "pre_request.semantic_guardrail.passed": {
            "$eq": false
          }
        },
        "message_mapper": {
          "modifier": "block",
          "content": "Your request contains content that violates our usage policies. Please review your message and try again with appropriate content."
        }
      },
      {
        "name": "rate_limit_exceeded_block",
        "conditions": {
          "pre_request.rate_limiter.passed": {
            "$eq": false
          }
        },
        "message_mapper": {
          "modifier": "block",
          "content": "You have exceeded your daily quota of 1,000 requests. Please try again tomorrow or upgrade to enterprise tier for higher limits."
        }
      },
      {
        "name": "fallback_route",
        "conditions": {
          "all": [
            {
              "pre_request.rate_limiter.passed": {
                "$eq": true
              }
            },
            {
              "pre_request.semantic_guardrail.passed": {
                "$eq": true
              }
            }
          ]
        },
        "targets": "openai/gpt-4o-mini"
      }
    ],
    "max_retries": 2
  }
}

Configuration Breakdown:

• Request Structure:
  • Uses "model": "router/dynamic" to enable dynamic routing
  • User information and compliance requirements are passed via the extra.user object
  • Router configuration includes pre_request interceptors and conditional routing
• Pre-Request Interceptors: Before routing evaluation, two interceptors run:
  • rate_limiter: Enforces 1,000 requests per day limit per user. Results are accessible via pre_request.rate_limiter.passed
  • semantic_guardrail: Scans content for policy violations. Results are accessible via pre_request.semantic_guardrail.passed (custom guardrail implementation required)
• Route 1: premium_eu_high_performance
  • Conditions: Requires premium tier (extra.user.tier), EU region (metadata.region), rate limit not exceeded, and low provider error rate
  • Targets: Routes to high-performance models with error rate filtering and minimum time-to-first-token sorting
• Route 2: gdpr_compliance_fallback
  • Conditions: Checks for GDPR compliance requirement via metadata.tags.gdpr (automatically set by LangDB from x-tags header)
  • Targets: Routes to specialized EU GDPR-compliant model regardless of user tier
• Route 3: semantic_guardrail_violation_block
  • Conditions: Triggers when semantic guardrail fails (pre_request.semantic_guardrail.passed = false)
  • Action: Uses message_mapper to block request and return policy violation error message
• Route 4: rate_limit_exceeded_block
  • Conditions: Triggers when rate limiter fails (pre_request.rate_limiter.passed = false)
  • Action: Uses message_mapper to block request and return informative error message with upgrade suggestion
• Route 5: fallback_route
  • Conditions: Only applies to requests that pass both rate limiting and semantic guardrail checks
  • Targets: Routes to openai/gpt-4o-mini as reliable fallback for valid requests that don't match other specific conditions

Key Features Demonstrated:

• Pre-Request Processing: Rate limiting and content guardrails before routing
• Compliance Routing: Automatic GDPR-compliant model selection
• Error Handling: Custom error messages for quota exceeded scenarios
• Performance Optimization: Provider health checks and fast response prioritization
• Regional Awareness: EU-specific routing based on metadata.region (automatically set by LangDB based on user location)