> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/core-concepts/evals.md).

# Evals

Agents can give different answers to the same request. Evals make quality measurable, repeatable, and reviewable.

Use evals to test changes before release and monitor quality after deployment.

#### At a glance

* **Before release:** Test known scenarios against expected answers.
* **In production:** Score live executions and route uncertain results for review.
* **Over time:** Add reviewed misses to your test dataset.

## Two evaluation loops

Qore evaluates agents before and after deployment.

| Loop                      | When it runs      | What it uses                          | Purpose                                          |
| ------------------------- | ----------------- | ------------------------------------- | ------------------------------------------------ |
| **Pre-release evals**     | Before deployment | Curated datasets and expected answers | Validate planned changes                         |
| **Production evaluation** | After deployment  | Live executions                       | Detect quality drift and route uncertain results |

Pre-release evals test known expectations. Production evaluation tests real traffic.

## Pre-release evals

In Agent Builder, an eval dataset contains test cases. Each case includes a question, expected answer, and optional documents.

Run the dataset after changing instructions, skills, knowledge, behavior, or guardrails. An LLM scorer assigns each response a score from zero to 10. Scores of seven or higher pass.

Include cases for:

* Frequent requests the agent must handle well
* Edge cases that expose unclear instructions or missing context
* Regressions found during production monitoring

Passing results provide evidence, not certainty. Test cases cannot cover every production request.

For setup and results, see [Evals](/agent-builder/test/evals.md).

## Production evaluation

After deployment, online evaluation scores live executions in the background. It doesn't delay or change responses.

In Control Center, an **evaluator** defines the quality method, judge model, and confidence threshold. Results below that threshold can route to **Human Review**.

A low-confidence result needs a person's judgment. It doesn't necessarily mean the agent failed.

Use production results to identify drift, recurring misses, and missing test cases. Turn reviewed findings into improved instructions, skills, knowledge, or learning cases.

For organization-wide evaluation and review routing, see [Evaluation & Quality](/governance/evaluation-and-quality.md). For per-agent production monitoring, see [Online-Eval](/agent-builder/monitor/online-eval.md).

## The improvement cycle

Use evaluation as a continuous control loop.

1. Define representative test cases and expected outcomes.
2. Run pre-release evals after each meaningful configuration change.
3. Deploy changes that meet your acceptance criteria.
4. Monitor live results.
5. Review uncertain outcomes.
6. Add reviewed production misses to the next test dataset.

{% hint style="warning" %}
Don't treat a model score as the final decision for high-impact workflows. Define review paths for outputs that affect customers, money, access, or compliance.
{% endhint %}

## How evals fit with other controls

Evals measure output quality. [Guardrails & Validation](/core-concepts/guardrails-and-validation.md) constrain what an agent can process or do. Neither replaces human review where judgment is required.

[Memory & Learning](/core-concepts/memory-and-learning.md) closes the loop. Reviewed production feedback becomes learning cases. Evals validate those improvements before broader rollout.

## Where this is configured

* [Evals](/agent-builder/test/evals.md)—create datasets, run tests, and review scored results
* [Evaluation & Quality](/governance/evaluation-and-quality.md)—define evaluators, inspect online results, and manage Human Review
* [Online-Eval](/agent-builder/monitor/online-eval.md)—monitor per-agent production scoring
* [Reinforcement Learnings](/agent-builder/monitor/reinforcement-learnings.md)—approve learning cases from reviewed feedback


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.uptiq.ai/core-concepts/evals.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.