Interceptors & Guardrails

Interceptors and Guardrails

Interceptors are custom logic that can run before or after a request is routed, allowing you to enrich, validate, or transform requests and responses. Guardrails are a common type of interceptor used to enforce policies.

Type	Purpose	Business Value	Example Configuration
pre_request	Analyze or enrich request	Classify topic, check for risk, personalize	rate_limiter, semantic_guardrail, toxicity_guardrail
post_request	Analyze or modify response	Moderate output, add fallback, redact sensitive info	fallback_response, content_moderator

Pre-Request Interceptor Configuration

Pre-request interceptors are defined in the routing.pre_request array and run before routing evaluation:

{
  "routing": {
    "pre_request": [
      { 
        "name": "rate_limiter", 
        "type": "interceptor", 
        "limit": 1000, 
        "period": "day", 
        "target": "user_id" 
      },
      { 
        "name": "semantic_guardrail", 
        "type": "guardrail" 
      }
    ]
  }
}

Interceptor Result Variables

When an interceptor runs, it injects its results into the routing context using the pre_request.{interceptor_name} namespace:

Result Variable Path	Description	Example Value	Business Use
pre_request.{interceptor_name}.passed	Whether the interceptor check passed	true	Conditional routing based on checks
pre_request.rate_limiter.passed	Rate limit check result	false	Enforce usage quotas and prevent abuse
pre_request.semantic_guardrail.passed	Content policy check result	true	Block policy violations
pre_request.toxicity_guardrail.passed	Toxicity check result	false	Block harmful content
pre_request.{guardrail}.result.topic	Detected topic from semantic analysis	"billing"	Route to topic-specialized models
pre_request.{guardrail}.result.score	Confidence or toxicity score	0.8	Threshold-based routing decisions
pre_request.{guardrail}.result.*	Any custom result from interceptor	Various	Custom business logic

Common Interceptor Types

Interceptor Type	Purpose	Configuration Fields	Result Fields
rate_limiter	Enforce request quotas	limit, period, target	passed
semantic_guardrail	Content classification and filtering	Custom guardrail configuration	passed, result.topic, result.*
toxicity_guardrail	Detect harmful content	Custom guardrail configuration	passed, result.score
custom_interceptor	Business-specific logic	Varies by implementation	passed, result.*

Usage in Routing Conditions

Use interceptor results in your routing conditions:

{
  "conditions": {
    "all": [
      { "pre_request.rate_limiter.passed": { "$eq": true } },
      { "pre_request.semantic_guardrail.passed": { "$eq": true } },
      { "pre_request.semantic_guardrail.result.topic": { "$eq": "support" } }
    ]
  }
}

Error Handling with Message Mappers

Block requests that fail interceptor checks:

{
  "name": "rate_limit_block",
  "conditions": {
    "pre_request.rate_limiter.passed": { "$eq": false }
  },
  "message_mapper": {
    "modifier": "block",
    "content": "Rate limit exceeded. Please try again later."
  }
}

Note on Custom Guardrails: Guardrails like semantic_guardrail and toxicity_guardrail are powerful examples of custom guardrails. Check out the Guardrails section for implementation details and configuration options.