> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/review-queue.md).

# Review Queue

Check the outputs a person needs to review, and record your verdict.

The Review Queue lists the outputs a person needs to check: failed evaluations, missed critical checks, and results the judge was unsure about. It covers every agent in the account. Open it in **Control Center → Evals → Review Queue**.

Select a task to read the judge's result, assign the task, and record your verdict. Select several tasks to assign or dismiss them together. **Review next** opens the next task, and the round-arrow button refreshes the queue.

## Permissions

| Action                            | Roles                                                                               |
| --------------------------------- | ----------------------------------------------------------------------------------- |
| View review tasks and take action | Approver, Developer, Governance Manager, Auditor, Account Admin, Organisation Admin |

See [Roles](https://docs.uptiq.ai/ai-workbench-qore/control-center/admin/roles.md).

## Reasons

The counts above the table show the tasks waiting for a verdict. **Unassigned** counts tasks with no assignee, and shows how many are assigned to you and how many are reviewed. **Critical check missed**, **Failed**, and **Judge unsure** count tasks by the reason they were routed here.

| Reason                    | Why a task is routed here                                     |
| ------------------------- | ------------------------------------------------------------- |
| **Failed**                | The evaluation's verdict was Fail.                            |
| **Critical check missed** | A checklist evaluator's critical check wasn't met.            |
| **Judge unsure**          | The judge's confidence was below the evaluator's review rule. |

A task can have more than one reason. Each evaluator has its own review rule, and review never changes the Pass or Fail result. The judge's verdict stays as scored.

## Filters

| Filter     | Options                                                                                                                                                                                                                                                                 |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Status** | **Needs review**, the default, lists tasks waiting for a verdict. **Assigned to me** lists tasks assigned to you. **Reviewed** lists tasks with a recorded verdict. **Dismissed** lists tasks closed without a verdict.                                                 |
| **Agent**  | One agent, or **All agents**, the default. Agents that share a name are told apart by their project and an ID.                                                                                                                                                          |
| **Reason** | **Critical check missed**, **Failed**, or **Judge unsure**, or **All reasons**, the default. Because a task can have more than one reason, the filter can return a task whose **Why it is here** shows a different one. The counts recalculate when you set **Reason**. |

Each list has a **Find an option…** box.

## Task columns

| Column             | Shows                                                                                                                              |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------- |
| **Priority**       | **High**, **Medium**, or **Low**.                                                                                                  |
| **Why it is here** | The reason the task was routed: **Failed**, **Critical check missed**, or **Judge unsure**.                                        |
| **Execution ID**   | The execution that was scored. An execution appears once for each evaluator that flagged it, so the same ID can fill several rows. |
| **Agent**          | The agent that produced the output.                                                                                                |
| **Evaluator**      | The evaluator that flagged it.                                                                                                     |
| **Score**          | The score the judge gave.                                                                                                          |
| **Confidence**     | How sure the judge is of its verdict.                                                                                              |
| **Assignee**       | The person the task is assigned to, or **Unassigned**.                                                                             |
| **Queued At**      | When the task entered the queue.                                                                                                   |

## Actions on several tasks

Select the checkbox on one or more rows, or the checkbox in the header to select the whole page. An action bar appears above the table with the number selected. Select **X** to clear the selection.

| Action               | Behavior                                                                                                                                                                                                                                |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Assign to me**     | Assigns the selected tasks to you. A toast confirms the update, and the counts include them under assigned to you.                                                                                                                      |
| **Dismiss**          | Closes the selected tasks without a verdict, with no confirmation. A toast confirms the update. The tasks move from **Needs review** to **Dismissed**, and each records who dismissed it and when. You can [reopen](#reopen-a-task) it. |
| **Agree with judge** | Records your agreement with each selected task at once. There is no form, so no reason or note is recorded. A toast confirms the update. Each task is then **Reviewed**, with **Judge: Agreed**.                                        |

## Task dialog

The dialog is headed by the execution ID and has these tabs.

| Tab              | Shows                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Overview**     | A **Please check** banner that says why the task is here. For a judge that is unsure, it reads "The judge wants a second opinion on these. It is a doubt, not a verdict." The banner lists the reason as a numbered item. For a checklist evaluator, it lists the checks that failed. Below it are the **Score** with the pass mark, the **Judge confidence** with the review rule, **Checks passed** for a checklist, the **Verdict**, **Agent**, **Evaluator**, **Execution ID**, **Queued** time, and **Assignee**. **Why it is here** and **Why a person reviews it** explain the routing, with the score, the pass mark, the confidence, and the review rule. |
| **Reasoning**    | One row for each judge. Open a row's **Actions** menu and select **View details** to see that judge's verdict, score, confidence, reasoning, and **Evidence**. It shows the same details as a run in [Online Eval Runs](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/online-eval-runs.md#open-a-run).                                                                                                                                                                                                                                                                                                                                              |
| **Checks**       | Each check of a checklist evaluator with its **Result** and **Weight**. A **Fail** or **Pass** comes with the judge's reason. It appears for checklist evaluators only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| **Conversation** | The user's message, the agent's reply, and the tools the agent used.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| **Assignment**   | Who reviews the task and how urgent it is.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |

In **Assignment**, the assignee list has every active user in the account, and **Unassigned**. The priority is **High**, **Medium**, or **Low**. **Save assignment** isn't available until you change something, and a toast confirms the change.

## Verdict buttons

Read the reasoning and the conversation, and then choose one of three buttons at the bottom of the dialog.

| Button               | Behavior                                                                                                                                                                                                                     |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Skip**             | Opens the next task without recording anything. The task you skipped stays in the queue, unchanged.                                                                                                                          |
| **Agree with judge** | You accept the judge's verdict, and the judge's human agreement goes up. The form reads "Agree with the judge: Failed" or "Agree with the judge: Passed," and "The judge's verdict stands, and its human agreement goes up." |
| **Override**         | Your verdict replaces the judge's, and the judge's human agreement goes down. The form flips the judge's result.                                                                                                             |

### Agree with judge fields

| Field                                | Behavior                                                                                                                                                                |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Closest reason**                   | Shown only when you agree with a failed result, and then required. Choose Wrong data, Wrong policy applied, Missing required flag, Hallucination, Formatting, or Other. |
| **What should change in the agent?** | Optional. Your note goes to [Reviewer Feedback](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/reviewer-feedback.md) for the team that owns the agent.    |

### Override fields

| The judge said | **Your verdict** | **Closest reason** you can choose                                                            |
| -------------- | ---------------- | -------------------------------------------------------------------------------------------- |
| **Passed**     | It fails         | Wrong data, Wrong policy applied, Missing required flag, Hallucination, Formatting, or Other |
| **Failed**     | It passes        | "Judge was wrong (agent was right)"                                                          |

**Your verdict** and **Closest reason** are required, and **Submit and next** isn't available until both are filled. **Back** returns to the task. The **What should change in the agent?** note is optional here too.

**Submit and next** records the verdict and opens the next task. Your verdict and reason appear in [Reviewer Feedback](https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/reviewer-feedback.md) under the agent they are about.

## Recorded tasks

A verdict or a dismissal moves the task out of **Needs review**. Open a reviewed task from **Reviewed** to see what was recorded.

| Item                               | Shows                                                                                                                            |
| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Banner                             | Reads "Recorded—your verdict closed this review." and shows your **Reviewer note**.                                              |
| **Reviewer verdict** and **Judge** | `Confirmed issue` and `Agreed` when you agreed with a failed result, or `Passed` and `Agreed` when you agreed with a passed one. |
| Details                            | The **Reason**, **Reviewed by**, and when it was **Reviewed**.                                                                   |

A dismissed task shows "Dismissed—no verdict was recorded." It sets **Review** to **Dismissed** and shows who dismissed it.

## Reopen a task <a href="#reopen-a-task" id="reopen-a-task"></a>

**Reopen** on a reviewed or dismissed task puts it back in the queue under **Needs review**, and its entry leaves Reviewer Feedback. A confirmation warns that the review returns to the queue as **Review Pending** and that this can't be undone. For a reviewed task, it adds that the recorded verdict is cleared from the evaluation. **Continue** reopens the task.

## Messages

| Message                                     | Meaning                          |
| ------------------------------------------- | -------------------------------- |
| `Verdict recorded`                          | The verdict saved.               |
| `Reopen this review?`                       | Confirms a reopen.               |
| "Recorded—your verdict closed this review." | The task has a recorded verdict. |
| "Dismissed—no verdict was recorded."        | The task was dismissed.          |

## What's next

<table data-view="cards"><thead><tr><th>Title</th><th>Description</th><th data-card-target data-type="content-ref">Link</th></tr></thead><tbody><tr><td><strong>Reviewer Feedback</strong></td><td>Read what reviewers say should change in an agent.</td><td><a href="https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/reviewer-feedback.md">Reviewer Feedback</a></td></tr><tr><td><strong>Online Eval Runs</strong></td><td>See every scored live run and how it was judged.</td><td><a href="https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/online-eval-runs.md">Online Eval Runs</a></td></tr><tr><td><strong>Evaluators</strong></td><td>Set the pass mark and the review rule for each evaluator.</td><td><a href="https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/evaluators.md">Evaluators</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.uptiq.ai/ai-workbench-qore/control-center/evals/review-queue.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
