> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/online-eval.md).
# Online Eval
Monitor evaluator scores and runs for live production executions.
Online Eval scores live production traffic. Open **Control Center → Evals → Online Eval**. The page has two tabs:
* **Scores** lists every score an evaluator produced, with its verdict and routing.
* **Runs** lists every evaluator run, including the ones that never produced a score, so you can see why a score is missing.
### Scores
The Scores tab.
Filter by project, agent, minimum score (**Score ≥**), verdict, or routing. Search by execution, agent, or evaluator.
The summary counts what the filters leave in view:
* **Evaluations in view**: every score that matches the filters.
* **Passed**: scores at or above the pass threshold, with their share of the total.
* **Routed to Human Review**: scores below judge confidence that were sent to a reviewer, with their share.
* **Avg. score**: the average score, with the average confidence beneath it.
Below the summary, the same counts are drawn as a flow from **Scored live** to **Passed** and **Routed to Human Review**. Select **Open board** to open [Human Review](/ai-workbench-qore/control-center/evals/human-review.md).
### Scored execution fields
| Column | What it shows |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Execution** | The execution ID. |
| **Agent** | The agent that produced the output. |
| **Evaluator** | The evaluator that scored it, such as *Jury Safety*. |
| **Score** | The result: `pass` or `fail` for a pass/fail method, or a percentage. |
| **Conf.** | The judge's confidence, with the evaluator's confidence threshold beneath it. `87%` over `≥ 95%` means a confidence of 87% against a threshold of 95%. |
| **Verdict** | **Passed** or **Failed**. |
| **Routing** | **Auto-accepted** or **Human Review**. |
| **Scored** | When the score was recorded, such as *7 Hrs ago*. |
{% hint style="info" %}
### Routing depends on confidence, not the verdict
An execution routes to Human Review when its confidence falls below the evaluator's threshold **and** the evaluator has **Route low-confidence results to human review** turned on (see [Evaluators](/ai-workbench-qore/control-center/evals/evaluators.md#create-an-evaluator)). This can happen even when the verdict is *Passed*.
A *Failed* execution is auto-accepted when confidence meets the threshold. Routing reflects judge confidence, not the verdict.
{% endhint %}
### Runs
When an agent's live conversation is sampled, Qore creates one run for each evaluator assigned to that agent. A run that scores shows up on **Scores**. A run that fails or is skipped has no score, so **Runs** is the only place it appears.
The Runs tab, one row per evaluator run.
Filter by project, agent, or status, and search by execution. Each row shows the **Execution**, **Agent**, **Evaluator**, **Status**, and when the run was **Created**.
**Pending** — the run is waiting to start.
**Running** — the judge model is scoring the conversation.
**Completed** — the run produced a score, which appears on **Scores**.
**Failed** — the run ended without a score. Hover over the status to see why:
* `Judge model unavailable: ` — the evaluator uses a judge model that isn't available.
* `Evaluation method not found` — the evaluator's method no longer exists.
* `Evaluation timed out` — the run stayed **Pending** or **Running** for more than two hours.
* `Evaluation failed` — scoring hit an error.
**Skipped** — the run was never scored, either because its evaluator was deleted before scoring started or because the conversation had nothing to score.
### Judge reasoning and evidence
Expand a score row to see the details behind it. **Judge reasoning** is the judge's explanation of the score. If the judge gave none, it reads *The judge did not return a justification for this score.*
Below it, **Evidence** lists up to three points behind the score, ordered from the one that moved it most to the one that moved it least. Each point cites something specific the agent did or failed to do, and says whether it raised or lowered the score. A judge can return fewer points, or none, and then no **Evidence** list is shown. For a **Jury** evaluator, each judge shows its own reasoning and evidence.
### Open the linked conversation
**Open linked conversation**, labelled with the run's execution ID, opens [Runs & Traces](/ai-workbench-qore/control-center/observe/runs-and-traces.md) filtered to that one conversation, so you can read exactly what the agent did. The filtered view survives a reload, so you can share its link with a colleague.
### Export results
Online Eval has no export button of its own. To download evaluation results, open **Control Center → Observe → Export** and choose the **Eval Results** dataset: scores, verdicts, reasoning, evidence, and human overrides for evaluated agent runs.
The Export page sets the project scope, date range, format, and delivery, and can run an export once or on a schedule. Large exports run in the background. See [Export](/ai-workbench-qore/control-center/observe/export.md).
### What the judge scores
Each evaluator scores a transcript of the conversation. The transcript includes:
* every message, with its formatting kept inside the turn it belongs to
* the documents attached by the user and produced by the agent, on each turn
* each tool call's arguments, whether it succeeded, and what it returned
* rich replies, such as widgets, rendered as text
Long tool results are shortened, and the transcript says where content was cut. A conversation too long for the judge model is trimmed to whole turns, and the transcript notes how many turns were left out.
This matters when you read a score. A judge sees what the tools returned, so an answer built on a failed tool call can score differently from the same answer built on a successful one.
***
{% columns %}
{% column width="83.33333333333334%" %}
Maintained by Abhishek Paul AI-assisted, human-approved
{% endcolumn %}
{% endcolumns %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/online-eval.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
