Use cases
Routing rules are most useful when traffic needs to be redirected or distributed at the gateway level based on request attributes, without modifying any calling application.Swap the model for a support bot without changing any code
Swap the model for a support bot without changing any code
Send simple and complex documents to different models
Send simple and complex documents to different models
Keep file attachment requests on models that support them
Keep file attachment requests on models that support them
Automatically reroute traffic when a provider goes down
Automatically reroute traffic when a provider goes down
Spread load evenly across multiple endpoints for the same model
Spread load evenly across multiple endpoints for the same model
How routing rules work
When a request arrives at the AI Gateway, every active rule whose CEL expression matches is a candidate. The winner is chosen in two steps:- Scope. Workspace-global rules outrank project-scoped rules. When any global rule matches, the winner comes from those whatever its priority, and project-scoped rules are considered only when no global rule matches.
- Priority. Within the winning scope, the lowest priority number wins, and the rule created first breaks a tie.
See also: Request-level load balancing
load_balancer parameter directly in your API calls.Creating a routing rule
From the Routing Rules list, click Routing Rule. The form opens as a full page with a back control and an unsaved-changes confirmation. It carries five sections, and the submit button reads Create Rule, or Save Changes once the rule exists.- AI Studio
- API & SDK
- CLI

The top of the Create Routing Rule form: the General fields and the Fallback load balancer with one target model.

The bottom of the Create Routing Rule form: the Plugins section and the Rule Builder with its generated CEL expression.
General
Load Balancer
Optional. Defines the target model or models to route matching requests to. Leave this section empty to build a rule that exists only to apply a plugin or a cache to matched traffic, with no model routing at all. Load balancing strategy sets how the targets are used:408, 429, 500, 502, 503, 504, and 529; leave it empty to keep the default. The API field is models_config.fallback_on_codes, accepting up to 20 unique integers from 400 through 599, and it applies in Fallback mode only.
Click Models to add a target model. The menu groups models by provider.
Each target has a remove control. Fallback also gives each target a drag handle for reordering, and Weighted adds a Traffic weight slider to set that model’s share, which must add up to 100%.
Cache
Optional. Give the rule its own response cache for matching requests. Exact-match requests reuse a cached response instead of hitting a model. Leaving this section disabled adds no cache of its own, so matched requests keep using their request-levelcache setting.
Set Enable cache to turn the cache on, then choose Time to live to control how long matching responses stay cached, from 5 minutes up to 3 days. The default is 1 hour.
cache parameter; the two are not merged.Plugins
Optional. Attach plugins that run on traffic matching this rule. Plugins can transform the request, the response, or stored traces. Click Plugin to attach one of the following:Conditions
Define when this routing rule should apply to a request. The Rule Builder builds the CEL expression shown read-only in the CEL Expression Preview; leaving it empty applies the rule to every request. Select Add rule to choose a condition type:- Request:
model,model_id,provider - Caller:
headers,metadata,identity,project,api_key
and by default; select the AND pill between two conditions to switch the group to or. Delete removes the whole group.
Rule groups are joined with or by default; select the pill between two groups to switch them to and.