Skip to main content
Routing Rules are -based conditions that intercept requests to the AI Gateway and redirect them to a different model when matched.

Use cases

Routing rules are most useful when traffic needs to be redirected or distributed at the gateway level based on request attributes, without modifying any calling application.
Intercepts support bot requests addressed to one model and silently redirects them to a cheaper one, so the calling application needs no changes to benefit from the cost saving.
Two rules at the same priority split document traffic by complexity: simple documents go to a lighter model, complex ones go to a more capable model, all based on metadata attached to each request.
Matches requests that include a file attachment and routes them through an ordered list of models that all support native file input, so the request succeeds even if the first model is unavailable.
A low-priority catch-all rule that redirects traffic to an alternative set of models whenever primary providers are unavailable, keeping requests flowing without manual intervention.
Distributes requests in round-robin mode across multiple endpoints serving the same model, balancing inference load across providers without any changes required on the caller side.

How routing rules work

When a request arrives at the AI Gateway, every active rule whose CEL expression matches is a candidate. The winner is chosen in two steps:
  1. Scope. Workspace-global rules outrank project-scoped rules. When any global rule matches, the winner comes from those whatever its priority, and project-scoped rules are considered only when no global rule matches.
  2. Priority. Within the winning scope, the lowest priority number wins, and the rule created first breaks a tie.
The winning rule supplies the target model it configures, its plugins, and its cache. The plugins and cache of the other matching rules never run, and a request that matches no rule keeps the model from its payload.

See also: Request-level load balancing

To distribute traffic across providers at the request level without organization-wide rules, use the load_balancer parameter directly in your API calls.

Creating a routing rule

From the Routing Rules list, click Routing Rule. The form opens as a full page with a back control and an unsaved-changes confirmation. It carries five sections, and the submit button reads Create Rule, or Save Changes once the rule exists.
Routing rules are visible to workspace administrators only.
Fill the five sections, then click Create Rule. The sections are described below the tabs.
Create Routing Rule form showing the rule name route-eu-support, priority 0, a disabled Enable rule switch, the Fallback strategy selected, claude-sonnet-4-6 as the only target model, and the Fallback status codes field.

The top of the Create Routing Rule form: the General fields and the Fallback load balancer with one target model.

Create Routing Rule form bottom showing an empty Plugins section and a Conditions section with a rule group titled Rule 1 · 1 condition holding one Metadata condition on region equals eu, the Condition and Add rule buttons, and the matching CEL expression metadata region equals eu.

The bottom of the Create Routing Rule form: the Plugins section and the Rule Builder with its generated CEL expression.

The five form sections below map to the request fields the API and CLI tabs send.

General

Load Balancer

Optional. Defines the target model or models to route matching requests to. Leave this section empty to build a rule that exists only to apply a plugin or a cache to matched traffic, with no model routing at all. Load balancing strategy sets how the targets are used: Fallback status codes appears once the rule has a target model. It overrides the default list of 408, 429, 500, 502, 503, 504, and 529; leave it empty to keep the default. The API field is models_config.fallback_on_codes, accepting up to 20 unique integers from 400 through 599, and it applies in Fallback mode only.
Choose Latency when minimizing response time matters more than a fixed traffic split. Choose Weighted or Round Robin when the split itself, such as cost control or A/B testing, is the goal.
Click Models to add a target model. The menu groups models by provider. Each target has a remove control. Fallback also gives each target a drag handle for reordering, and Weighted adds a Traffic weight slider to set that model’s share, which must add up to 100%.
When a request matches this routing rule, its Load Balancer configuration completely replaces the request’s own model, load_balancer, fallbacks, and fallback_on_codes values; they are never merged. A request-level load_balancer parameter has no effect once a matching routing rule with configured models applies.

Cache

Optional. Give the rule its own response cache for matching requests. Exact-match requests reuse a cached response instead of hitting a model. Leaving this section disabled adds no cache of its own, so matched requests keep using their request-level cache setting. Set Enable cache to turn the cache on, then choose Time to live to control how long matching responses stay cached, from 5 minutes up to 3 days. The default is 1 hour.
When a rule has cache enabled, a matching request uses the rule’s cache configuration instead of the request’s own cache parameter; the two are not merged.
See LLM response cache for how caching works.

Plugins

Optional. Attach plugins that run on traffic matching this rule. Plugins can transform the request, the response, or stored traces.
Only the winning rule runs its plugins. A project-scoped rule that carries only a plugin never runs while any workspace-global rule matches. To redact per project, attach the PII Redaction plugin to the global rule and match the project in its condition.
Click Plugin to attach one of the following:

Conditions

Define when this routing rule should apply to a request. The Rule Builder builds the CEL expression shown read-only in the CEL Expression Preview; leaving it empty applies the rule to every request. Select Add rule to choose a condition type: The CEL expression the Rule Builder generates reads the request through these variables:
  • Request: model, model_id, provider
  • Caller: headers, metadata, identity, project, api_key
Each Add rule creates a condition group titled Rule 1 · N conditions. Inside a group, select Condition to add another condition, and to remove one. Drag a condition by its handle to reorder it. Conditions in a group are joined with and by default; select the AND pill between two conditions to switch the group to or. Delete removes the whole group. Rule groups are joined with or by default; select the pill between two groups to switch them to and.

Editing and deleting a routing rule

Click a rule’s name in the Routing Rules list to open it in the same form. An existing rule exposes Delete Rule in the form footer, next to Save Changes. Deleting asks for confirmation first. The list’s row actions menu offers Edit, Enable or Disable, and Delete.