> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/ai-workbench-qore/control-center/ai-gateway/how-model-routing-works.md).
# How Model Routing Works
Understand families, providers, gateways, credentials, and tiers before you configure them.
Every model request an agent makes passes through AI Gateway. The gateway selects a family, resolves a tier to a model, and sends the request through an available provider.
Read this page before you open [Gateway Settings](/ai-workbench-qore/control-center/ai-gateway/gateway-settings.md). The screens there expose one field per concept below, and the fields make more sense once the routing model is clear.
## Routing building blocks
{% columns %}
{% column %}
#### Model and family
A **model** is an individual endpoint, such as *Claude Sonnet 4.5* or *GPT 4o*.
A **model family** groups related models from one vendor. Configuration applies at the family level, not to individual models.
{% endcolumn %}
{% column %}
#### Provider and gateway
A **provider** hosts and serves a model.
A **gateway** routes a family's requests to its configured providers and applies authentication and failover.
{% endcolumn %}
{% endcolumns %}
A **tier** sits on top: it names a capability level, and each family maps its own model to each tier.
## Model families
Enable only the families your organization approves. One enabled family is the default and handles any request that doesn't name a family.
| Family | What it offers |
| ------------- | --------------------------------------------------------------------------------- |
| **Anthropic** | Claude Sonnet, Opus, and Haiku. Reasoning, long context, and vision. |
| **Gemini** | Gemini Pro and Flash. Long context, fast, low-cost inference. Default as shipped. |
| **OpenAI** | GPT 4o, GPT 4.1, GPT 5.x, and the o-series. General reasoning and tool use. |
{% hint style="info" %}
Exactly one enabled family is the default. Setting a new default removes the previous one.
{% endhint %}
**Gemini and Google name the same family.** [Gateway Settings](/ai-workbench-qore/control-center/ai-gateway/gateway-settings.md) labels it Gemini; [Model Catalog](/ai-workbench-qore/control-center/ai-gateway/model-catalog.md) labels it Google. The underlying family is the same.
## Providers
A family can use more than one provider. When several providers are enabled for a family, the gateway has more than one path to the same model.
| Family | Available providers |
| --------- | --------------------------------------- |
| Anthropic | Anthropic direct; GCP Model Garden |
| Gemini | Google AI Studio; Google Agent Platform |
| OpenAI | OpenAI direct; Azure OpenAI |
The provider you choose decides more than availability. It decides which cloud account runs the inference, which region it runs in, and who the provider bills.
## Gateways and credentials
{% tabs %}
{% tab title="UPTIQ AI Gateway" %}
The default gateway. It provides managed routing, retries, caching, failover, and observability. Configure each provider in the family with a **UPTIQ managed key** or **Bring your own key**.
{% endtab %}
{% tab title="OpenRouter" %}
An alternative gateway that uses your organization's own OpenRouter API key. Enter the key, then fetch the models enabled on that account. Requests for the selected family are proxied through OpenRouter to whichever upstream serves the model.
{% endtab %}
{% endtabs %}
| Credential option | Who supplies it | Who is billed |
| ---------------------- | ----------------- | ------------------------------- |
| **UPTIQ managed key** | The platform | Your credit balance |
| **Bring your own key** | Your organization | The provider bills you directly |
| **OpenRouter key** | Your organization | OpenRouter bills you directly |
A single family can mix managed and bring-your-own-key providers under **UPTIQ AI Gateway**.
{% hint style="warning" %}
OpenRouter chooses the upstream provider itself, so you don't control which one serves a request or where it runs. Where region residency matters, use **UPTIQ AI Gateway** with your own key against your own provider tenancy instead.
{% endhint %}
## Model tiers
An application requests a **tier**, not a model name. The tier resolves to whichever model the family currently maps to it.
This is the point of the whole arrangement. An administrator can move the Smart tier from one model to a newer one, and every agent that asks for Smart picks up the change — no agent is edited, and no model name is hard-coded anywhere.
{% columns %}
{% column %}
#### Economy
Routing, classification, extraction, and reranking. Prioritizes speed and cost for high-volume work.
{% endcolumn %}
{% column %}
#### Smart
Synthesis, specialist tasks, reports, and drafts. The balanced choice for most agents.
{% endcolumn %}
{% column %}
#### Premium
The hardest reasoning. Prioritizes quality over latency and cost.
{% endcolumn %}
{% endcolumns %}
{% hint style="warning" %}
Nothing stops you from mapping all three tiers to the same model, and the product doesn't warn when you do. Choose distinct models where you want a real difference in cost or capability.
{% endhint %}
## What routes through the gateway
An agent sends every model call through AI Gateway: the conversation itself and the smaller calls it makes behind the scenes. Those background calls ask for a tier, not a model, so remapping a tier in [Gateway Settings](/ai-workbench-qore/control-center/ai-gateway/gateway-settings.md) changes them too.
| Call | Model used |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Planning and execution | The agent's [Planning Model and Execution Model](/ai-workbench-qore/console/agent-builder/build/agent-behavior.md#advanced-controls) |
| Knowledge queries | Economy tier when the agent's **Response Mode** is **Flash**; Smart tier when it's **Balanced** |
| Document summaries | The agent's execution model, or the Economy tier if it has none |
| Document classification | Economy tier |
| Conversation titles | Economy tier |
Because the gateway handles every one of these calls, it also handles their failover. See [When a call can't be served first time](#when-a-call-cant-be-served-first-time).
### Model families for agents
An agent's **Model Family** field lists the families the gateway serves for your organization. Agents can use the **Google** (Gemini) and **OpenAI** families. Other families, such as Anthropic, aren't offered for agents in production yet. Where they appear, they're marked **Dev only**.
The agent builder only offers what your deployment can serve:
* A family the gateway can't serve isn't listed.
* A model the gateway can't serve is greyed out in the **Planning Model** and **Execution Model** lists.
* A new agent starts on models the gateway can serve. If the default family isn't available, it starts on one that is.
* A saved family or model that's no longer available stays visible, marked **Unavailable**, so the field is never blank.
## How a request routes
```
Application request
→ named family, or the default family
→ Economy, Smart, or Premium tier
→ the model mapped to that tier
→ the family's gateway
→ an eligible provider
→ model response
```
A tier that maps to a model whose provider isn't configured fails at request time. Confirm availability in [Model Catalog](/ai-workbench-qore/control-center/ai-gateway/model-catalog.md) before you assign a tier.
### When a call can't be served first time
The gateway works through a ranked list of candidates: each eligible provider for the mapped model, then the family's other models in fallback order.
* **Rate limits and provider errors.** If a provider is rate-limited, times out, or returns a server error, the gateway retries, then moves on to the next candidate.
* **Outages.** A model and provider pair that keeps failing is treated as in outage and skipped. It's used again once it recovers.
* **When every candidate fails.** The agent tells the user, in plain words, whether the provider was rate-limited or unavailable, rather than showing a generic error.
#### Requests for JSON or a response schema
A call can ask for a JSON reply, or for a reply that matches a fixed schema. The gateway tries the models that can honour that format first. If none of the remaining candidates can, the call still goes to the best one left, with the format dropped, rather than failing.
{% hint style="warning" %}
**A reply isn't guaranteed to be JSON.** The requested format is a preference, not a guarantee, so validate the shape of the reply in your own code. Today, only OpenAI and Google models enforce a response schema.
{% endhint %}
## Native on Cloud
Native on Cloud keeps model hosting and inference inside a cloud tenancy your organization controls. Route through GCP Model Garden, Google Agent Platform, or Azure OpenAI where the deployment must make no calls to public AI APIs.
Region is set on the provider credential rather than centrally in the gateway, because it's the provider resource that's region-bound — an Azure OpenAI endpoint belongs to the region its resource was created in, and a GCP credential carries its project and region.
For how this works in a bank-owned deployment, see [Security FAQ](/ai-workbench-qore/control-center/detailed-security-and-runtime-faq.md).
## Why this matters
* **Configuration is per family.** Families are enabled, authenticated, and tier-mapped one at a time.
* **Applications ask for a tier.** Change the model behind a tier without touching an agent.
* **Agents' background calls use tiers too.** Classification, titles, summaries, and knowledge queries follow your tier mappings.
* **The gateway owns failover.** A rate-limited or failing provider is retried, then skipped for the next candidate.
* **The provider decides where inference runs.** Choose it deliberately when region matters.
* **One family is the default.** It serves every request that doesn't name one.
## Related
| Title | Description | Link |
|---|
| Gateway Settings | Enable families, authenticate providers, and map tier models. | Gateway Settings |
| Model Catalog | Check which models and providers are available before assigning a tier. | Model Catalog |
| Credit Usage | See what managed-key usage draws from your credit balance. | Credit Usage |
| Security FAQ | Where the runtime and its data sit in a bank-owned deployment. | Security FAQ |
| Glossary | Look up family, provider, gateway, tier, and credential terms. | Glossary |
***
{% columns %}
{% column width="83.33333333333334%" %}
Maintained by Abhishek Paul
AI-assisted, human-approved
{% endcolumn %}
{% column width="16.666666666666664%" %}
{% endcolumn %}
{% endcolumns %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://docs.uptiq.ai/ai-workbench-qore/control-center/ai-gateway/how-model-routing-works.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
