How Model Routing Works
Understand families, providers, gateways, credentials, and tiers before you configure them.
Every model request an agent makes passes through AI Gateway. The gateway selects a family, resolves a tier to a model, and sends the request through an available provider.
Read this page before you open Gateway Settings. The screens there expose one field per concept below, and the fields make more sense once the routing model is clear.
Routing building blocks
A tier sits on top: it names a capability level, and each family maps its own model to each tier.
Model families
Enable only the families your organization approves. One enabled family is the default and handles any request that doesn't name a family.
Anthropic
Claude Sonnet, Opus, and Haiku. Reasoning, long context, and vision.
Gemini
Gemini Pro and Flash. Long context, fast, low-cost inference. Default as shipped.
OpenAI
GPT 4o, GPT 4.1, GPT 5.x, and the o-series. General reasoning and tool use.
Gemini and Google name the same family. Gateway Settings labels it Gemini; Model Catalog labels it Google. The underlying family is the same.
Providers
A family can use more than one provider. When several providers are enabled for a family, the gateway has more than one path to the same model.
Anthropic
Anthropic direct; GCP Model Garden
Gemini
Google AI Studio; Google Agent Platform
OpenAI
OpenAI direct; Azure OpenAI
The provider you choose decides more than availability. It decides which cloud account runs the inference, which region it runs in, and who the provider bills.
Gateways and credentials
The default gateway. It provides managed routing, retries, caching, failover, and observability. Configure each provider in the family with a UPTIQ managed key or Bring your own key.
An alternative gateway that uses your organization's own OpenRouter API key. Enter the key, then fetch the models enabled on that account. Requests for the selected family are proxied through OpenRouter to whichever upstream serves the model.
UPTIQ managed key
The platform
Your credit balance
Bring your own key
Your organization
The provider bills you directly
OpenRouter key
Your organization
OpenRouter bills you directly
A single family can mix managed and bring-your-own-key providers under UPTIQ AI Gateway.
OpenRouter chooses the upstream provider itself, so you don't control which one serves a request or where it runs. Where region residency matters, use UPTIQ AI Gateway with your own key against your own provider tenancy instead.
Model tiers
An application requests a tier, not a model name. The tier resolves to whichever model the family currently maps to it.
This is the point of the whole arrangement. An administrator can move the Smart tier from one model to a newer one, and every agent that asks for Smart picks up the change — no agent is edited, and no model name is hard-coded anywhere.
Nothing stops you from mapping all three tiers to the same model, and the product doesn't warn when you do. Choose distinct models where you want a real difference in cost or capability.
What routes through the gateway
An agent sends every model call through AI Gateway: the conversation itself and the smaller calls it makes behind the scenes. Those background calls ask for a tier, not a model, so remapping a tier in Gateway Settings changes them too.
Planning and execution
The agent's Planning Model and Execution Model
Knowledge queries
Economy tier when the agent's Response Mode is Flash; Smart tier when it's Balanced
Document summaries
The agent's execution model, or the Economy tier if it has none
Document classification
Economy tier
Conversation titles
Economy tier
Because the gateway handles every one of these calls, it also handles their failover. See When a call can't be served first time.
Model families for agents
An agent's Model Family field lists the families the gateway serves for your organization. Agents can use the Google (Gemini) and OpenAI families. Other families, such as Anthropic, aren't offered for agents in production yet. Where they appear, they're marked Dev only.
The agent builder only offers what your deployment can serve:
A family the gateway can't serve isn't listed.
A model the gateway can't serve is greyed out in the Planning Model and Execution Model lists.
A new agent starts on models the gateway can serve. If the default family isn't available, it starts on one that is.
A saved family or model that's no longer available stays visible, marked Unavailable, so the field is never blank.
How a request routes
A tier that maps to a model whose provider isn't configured fails at request time. Confirm availability in Model Catalog before you assign a tier.
When a call can't be served first time
The gateway works through a ranked list of candidates: each eligible provider for the mapped model, then the family's other models in fallback order.
Rate limits and provider errors. If a provider is rate-limited, times out, or returns a server error, the gateway retries, then moves on to the next candidate.
Outages. A model and provider pair that keeps failing is treated as in outage and skipped. It's used again once it recovers.
When every candidate fails. The agent tells the user, in plain words, whether the provider was rate-limited or unavailable, rather than showing a generic error.
Requests for JSON or a response schema
A call can ask for a JSON reply, or for a reply that matches a fixed schema. The gateway tries the models that can honour that format first. If none of the remaining candidates can, the call still goes to the best one left, with the format dropped, rather than failing.
A reply isn't guaranteed to be JSON. The requested format is a preference, not a guarantee, so validate the shape of the reply in your own code. Today, only OpenAI and Google models enforce a response schema.
Native on Cloud
Native on Cloud keeps model hosting and inference inside a cloud tenancy your organization controls. Route through GCP Model Garden, Google Agent Platform, or Azure OpenAI where the deployment must make no calls to public AI APIs.
Region is set on the provider credential rather than centrally in the gateway, because it's the provider resource that's region-bound — an Azure OpenAI endpoint belongs to the region its resource was created in, and a GCP credential carries its project and region.
For how this works in a bank-owned deployment, see Security FAQ.
Why this matters
Configuration is per family. Families are enabled, authenticated, and tier-mapped one at a time.
Applications ask for a tier. Change the model behind a tier without touching an agent.
Agents' background calls use tiers too. Classification, titles, summaries, and knowledge queries follow your tier mappings.
The gateway owns failover. A rate-limited or failing provider is retried, then skipped for the next candidate.
The provider decides where inference runs. Choose it deliberately when region matters.
One family is the default. It serves every request that doesn't name one.
Related
Maintained by Abhishek Paul
AI-assisted, human-approved
Last updated
Was this helpful?