> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/evals-faqs-and-troubleshooting.md).

# Evals: FAQs & Troubleshooting

Answers to common Evals questions, and fixes for missing scores, failed runs, and empty review queues.

This page answers common questions about evaluators, Online Eval, and Human Review. Use the troubleshooting entries when a score is missing, a run fails, or a review task doesn't reach anyone.

## FAQs

### Evaluators

<details>

<summary>Who can create evaluators and see results?</summary>

**Build evals**: Developer, Governance Manager, Account Admin, and Organisation Admin. **View online evals**: Approver, Developer, Governance Manager, Auditor, Account Admin, and Organisation Admin. See [Roles](/ai-workbench-qore/control-center/admin/roles.md).

</details>

<details>

<summary>Is an evaluator created per agent?</summary>

No. Evaluators are defined at the account level. Use **Agent enablement** to choose which agents each evaluator scores, so one evaluator can serve every agent that enables it. See [Evaluators](/ai-workbench-qore/control-center/evals/evaluators.md).

</details>

<details>

<summary>What's the difference between Single judge and Jury?</summary>

**Single judge** scores each output with one model. **Jury** scores it with 2 to 5 judge models, which can come from different providers, and combines their votes into one weighted verdict. See [Choose a framework](/ai-workbench-qore/control-center/evals/evaluators.md#choose-a-framework).

</details>

<details>

<summary>Which evaluation methods are there?</summary>

**Answer Relevancy**, **Faithfulness**, **Safety**, and **Task Completion**. See [Create an evaluator](/ai-workbench-qore/control-center/evals/evaluators.md#create-an-evaluator).

</details>

<details>

<summary>What does Faithfulness measure?</summary>

How much of what the agent said is supported by the context it was given: retrieved context, tool results, and what the user said. It judges support, not usefulness or correctness, so a true claim that isn't in the context counts as unsupported. See [Faithfulness](/ai-workbench-qore/control-center/evals/evaluators.md#faithfulness).

</details>

<details>

<summary>What does the judge see?</summary>

A transcript of the conversation: every message, the documents on each turn, each tool call's arguments and results, and rich replies rendered as text. Long tool results are shortened, and a conversation too long for the judge is trimmed to whole turns. See [What the judge scores](/ai-workbench-qore/control-center/evals/online-eval.md#what-the-judge-scores).

</details>

### Scores and review

<details>

<summary>Why did a passed result go to Human Review?</summary>

Routing depends on confidence, not the verdict. A result routes to Human Review when the judge's confidence is below the evaluator's threshold and the evaluator has **Route low-confidence results to human review** on. A failed result with confidence at or above the threshold is auto-accepted. See [Online Eval](/ai-workbench-qore/control-center/evals/online-eval.md#scored-execution-fields).

</details>

<details>

<summary>Is every conversation scored?</summary>

Only the sampled share. Each agent's **Sampling** sets the chance that a live conversation is evaluated; it defaults to `100%`. See [Agent enablement](/ai-workbench-qore/control-center/evals/evaluators.md#agent-enablement).

</details>

<details>

<summary>Does scoring cost credits?</summary>

Yes. Online Eval scoring is billed on its own, as `OnlineEval` on the **Features** tab of Credit Usage, and isn't counted in the agent's own row. Lower **Sampling** on high-traffic agents to keep judge costs down. See [Credit Usage](/ai-workbench-qore/control-center/cost-control/credit-usage.md#review-credit-usage).

</details>

<details>

<summary>How are Human Review tasks assigned?</summary>

Through teams, not directly to people. A reviewer who isn't on a team has no assigned tasks. See [Human Review](/ai-workbench-qore/control-center/evals/human-review.md).

</details>

<details>

<summary>How do I export evaluation results?</summary>

Online Eval has no export button. Open **Observe** › **Export** and choose the **Eval Results** dataset: scores, verdicts, reasoning, evidence, and human overrides. See [Export results](/ai-workbench-qore/control-center/evals/online-eval.md#export-results).

</details>

## Troubleshooting

### Missing scores

<details>

<summary>An agent's conversations aren't being scored</summary>

Check all three: evaluators are assigned to the agent, the agent's **Online eval** toggle is on, and **Sampling** isn't set too low. Assigning evaluators doesn't start scoring on its own. See [Assign evaluators to an agent](/ai-workbench-qore/control-center/evals/evaluators.md#assign-evaluators-to-an-agent).

</details>

<details>

<summary>An agent shows Unsupported instead of the Online eval toggle</summary>

That agent's type doesn't support Online Eval yet. You can still pre-assign evaluators with **Edit**. See [Agent enablement](/ai-workbench-qore/control-center/evals/evaluators.md#agent-enablement).

</details>

<details>

<summary>A score is missing for a conversation</summary>

Open the **Runs** tab, which lists every run, including those that produced no score. A **Failed** or **Skipped** run never appears on **Scores**. See [Runs](/ai-workbench-qore/control-center/evals/online-eval.md#runs).

</details>

<details>

<summary>A run shows Failed</summary>

Hover over the status for the reason:

* `Judge model unavailable: <model>`: the evaluator's judge model isn't available.
* `Evaluation method not found`: the evaluator's method no longer exists.
* `Evaluation timed out`: the run stayed **Pending** or **Running** for more than two hours.
* `Evaluation failed`: scoring hit an error.

See [Runs](/ai-workbench-qore/control-center/evals/online-eval.md#runs).

</details>

<details>

<summary>A run shows Skipped</summary>

Its evaluator was deleted before scoring started, or the conversation had nothing to score. See [Runs](/ai-workbench-qore/control-center/evals/online-eval.md#runs).

</details>

<details>

<summary>A score has no reasoning or evidence</summary>

The judge returned none. Reasoning then reads *The judge did not return a justification for this score.*, and no **Evidence** list is shown. See [Judge reasoning and evidence](/ai-workbench-qore/control-center/evals/online-eval.md#judge-reasoning-and-evidence).

</details>

### Evaluators and review

<details>

<summary>New evaluator is disabled</summary>

Hovering shows *You don't have access to create or change evaluators*. Creating an evaluator needs the **Build evals** capability: Developer, Governance Manager, Account Admin, or Organisation Admin. See [Create an evaluator](/ai-workbench-qore/control-center/evals/evaluators.md#create-an-evaluator).

</details>

<details>

<summary>Create stays disabled in the new evaluator dialog</summary>

Set a name, a method, and the judge model, or 2 to 5 jury models. See [Create an evaluator](/ai-workbench-qore/control-center/evals/evaluators.md#create-an-evaluator).

</details>

<details>

<summary>Low-confidence results never reach Human Review</summary>

The evaluator's **Route low-confidence results to human review** is off by default. While it's off, low-confidence results are scored and auto-accepted. Turn it on in the evaluator. See [Create an evaluator](/ai-workbench-qore/control-center/evals/evaluators.md#create-an-evaluator).

</details>

<details>

<summary>A reviewer sees no tasks under My tasks</summary>

Tasks are assigned through teams. Add the reviewer to a team in **Admin** › **Teams**, or check **Unassigned**. See [Human Review](/ai-workbench-qore/control-center/evals/human-review.md).

</details>

### Related

* [Evaluators](/ai-workbench-qore/control-center/evals/evaluators.md), [Online Eval](/ai-workbench-qore/control-center/evals/online-eval.md), and [Human Review](/ai-workbench-qore/control-center/evals/human-review.md)
* [Evaluate agent quality](/ai-workbench-qore/task-guides/evaluate-agent-quality.md): every Evals task
* [Evals](/ai-workbench-qore/console/agent-builder/test/evals.md) in Agent Builder: test datasets before release

***

{% columns %}
{% column width="83.33333333333334%" %}

<p align="right"><em>Maintained by</em> <mark style="color:green;">Abhishek Paul</mark><br><code>AI-assisted, human-approved</code></p>
{% endcolumn %}

{% column width="16.666666666666664%" %}

<div align="left"><img src="https://1326225582-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F0qmgQjJ5aArDTj2ACFHG%2Fuploads%2Fgit-blob-d034645b4f8f8ee661985f5306b60ded3289653c%2Fmaintainer-abhishek-paul.png?alt=media" alt="" width="60"></div>
{% endcolumn %}
{% endcolumns %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/evals-faqs-and-troubleshooting.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.