Skip to main content

Routing with Virtual Model

In LangDB AI Gatewau, any virtual model can act as a router. Just define a strategy and list of target models—it’ll route requests based on metrics like cost, latency, percentage, er or custom rules.

Setting up Routing​

Setting up routing in a virtual model is straightforward:

  1. Open any virtual model in the Chat Playground and click Show Config
  2. Choose a routing strategy (like fallback, optimized, percentage, etc.)
  3. Add your target models—each one can be configured just like the virtual models you set up in the previous section

Each target defines:

  • Which model to use
  • Prompt
  • MCP Servers
  • Guardrails
  • Response Format
  • Custom parameters like temperature, max_tokens, penalties, etc.

All routing options are available directly in the virtual model config panel.

Check more about the Routing Strategies.