> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/agent-builder/monitor/online-eval.md).

# Online-Eval

Online Evals is the production-side evaluation system on the Uptiq Platform. Once an agent is deployed, it runs by itself on real conversations, without anyone asking it to. The grading happens off to the side — not in the path of the actual response. If the grading itself fails for any reason — a timeout, an error — that failure gets logged and retried, but it never touches the live conversation. The user never sees it, never feels it.

| Capability              | Description                                                                  |
| ----------------------- | ---------------------------------------------------------------------------- |
| Background scoring      | Runs automatically on completed conversations, off the critical path         |
| Configurable scorers    | Define which quality dimensions are evaluated, with custom prompts           |
| Performance dashboard   | Current scores, trends over time, and drill-down to specific conversations   |
| Threshold alerting      | Get alerted when quality drops past a threshold you've set                   |
| Feedback loop           | Production problems feed back into the next round of building, done manually |
| Failure isolation       | Grading failures are logged and retried — never touch the live conversation  |
| Model-provider agnostic | Works regardless of which LLM powers the agent or the grader                 |

{% hint style="info" %}
Online Evals is the runtime counterpart to [Evals](/agent-builder/test/evals.md). Use Evals to validate quality before deployment, Online Evals to score live traffic after deployment, and [Reinforcement Learnings](/agent-builder/monitor/reinforcement-learnings.md) to turn reviewed feedback into approved learning cases.
{% endhint %}

Online Evals is accessed from the **Cognition & Control** section in the Agent Builder's "Manage Agent" modal sidebar. The interface has two tabs:

* **Scorers** — Configure which quality dimensions are evaluated
* **Runs** — View evaluation run history and results

***

## How It Works

Once the agent is live, Online Evals takes over automatically:

1. End users invoke the agent in real conversations
2. Online eval runs in the background — non-intrusive
3. Performance results stream to the Maintainer's dashboard
4. Maintainer tracks agent performance over time
5. **Performance drift or regression?**
   * **No** → Keep watching
   * **Yes** → Iterate and refine the agent — feeds back to the builder

If something's drifted, that loops back to whoever built the agent, so production problems actually feed back into the next round of building — not just into a report nobody reads.

***

## Dashboard

What comes out of Online Evals lands on a dashboard for whoever's maintaining the agent:

* **Current performance** — How the agent is scoring right now
* **Threshold suggestions** — If quality drops past a threshold score, suggestions pop up
* **Drill-down** — Click into a metric and see exactly which real conversations are dragging the score down

***

## Guardrails and Governance

Who can do what is enforced at the platform level — it's a permission boundary, not a process people are trusted to follow. Builders get the offline/testing side, Maintainers get the live monitoring side.

Whatever real conversation data feeds the online evals still respects tenant isolation and PII rules — it's not an open pipe into live data.

Online Evals works the same regardless of which LLM provider is behind either the agent or the grader.

### Related sections

* [Monitoring](broken://pages/cD46XWa7ZwMLNvtnC3yG) — Track production health and investigate runtime issues.
* [Evals](/agent-builder/test/evals.md) — Validate changes against test datasets before rollout.
* [Reinforcement Learnings](/agent-builder/monitor/reinforcement-learnings.md) — Review approved corrections from real conversations.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.uptiq.ai/agent-builder/monitor/online-eval.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.