Agent Evaluation Plan: [Agent name] [version]

Evaluation plan for a tool-using AI agent: objectives, task suite design, metrics for task success, trajectory quality, tool-call and argument accuracy, efficiency, safety and recovery, the offline harness and online evaluation, simulated environments and sandboxing, graders (programmatic, LLM-as-judge with calibration, human), statistical method, regression gates, adversarial cases linked to the OWASP Top 10 for Agentic Applications 2026 (ASI01 to ASI10), and reporting. It covers multi-step, multi-turn behaviour; use an LLM evaluation plan for single-turn quality of answers. Replace every [bracketed] value and every row marked Example. Worked examples use a travel booking agent that searches flights, holds seats, books with a stored payment method and handles changes.

Document control

Approvals

Revision history

Contents

1. Objectives and scope
2. Task suite design
3. Metrics
4. Environments and sandboxing
5. Offline harness
6. Online evaluation
7. Graders
8. Statistical method
9. Safety and adversarial cases
10. Regression gates
11. Reporting
12. Cadence and ownership
13. Glossary and references

1. Objectives and scope

1.1 Agent summary

[Agent name] is a [travel booking assistant] that [searches flights, holds seats, books with a stored payment method, changes and cancels bookings]. It runs at autonomy level [2]: it acts alone on searches and holds, and asks the user to confirm any payment or cancellation.

1.2 Evaluation objectives

1.3 Out of scope

• Single-turn answer quality such as tone and factual accuracy of travel advice; covered by the [LLM evaluation plan].
• Load and capacity testing of the tool APIs.
• Manual red teaming; covered by the [AI red team plan and report], which receives the adversarial cases in section 9 as a starting point.

2. Task suite design

2.1 Task anatomy

Each task has an ID (AEP-TASK-NNN), a user goal, a starting environment state, a simulated user persona (for multi-turn tasks), the expected end state, the acceptable trajectories, forbidden actions and the grader that decides success.

2.2 Task categories and coverage

2.3 Task quality rules

• Every task has one unambiguous success definition that two reviewers agree on before it enters the suite.
• Hold out [20%] of tasks as a private set that prompt authors never see, to detect overfitting to the visible set.
• Version the suite (suite-vN); never edit a task in place after it has been used for a gate. Retire and replace instead.
• Record known-unsolvable tasks so a correct refusal counts as success.

3. Metrics

3.1 Metric definitions

3.2 Repeated-run metrics

Run each task $n$ times. With $c$ successful runs out of $n$, the per-task success rate is $c / n$. The probability that a user who tries up to $k$ times sees at least one success is estimated by $\text{pass@}k$, while the probability that all $k$ attempts succeed (a reliability measure) is estimated by pass-all-$k$, written $\text{pass}^{k}$ (with $k \le n$):

where $n$ is the number of runs drawn for the task and $c$ the successes among them. For agents that act with side effects, report pass-all-$k$: users experience every run, not only the best one.

3.3 Trajectory rubric

4. Environments and sandboxing

Sandbox requirements:

• Runs are isolated: one run cannot see another run's state, memory or holds.
• Agent credentials in evaluation have no access to production systems.
• Code execution, if any, runs in a container with no outbound network and time and memory limits.
• The simulated user follows the persona script and is itself versioned; a change to the user simulator is a change to the suite.
• Fault injection (errors, timeouts, malformed responses, rate limits) is configured per task and logged with the run.

5. Offline harness

Run procedure:

1. Freeze the agent bundle and record its versions in Document control.
2. Reset the environment to the task's start state.
3. Run the task $n$ times with recorded seeds.
4. Apply graders in section 7 to each trace and end state.
5. Aggregate metrics with intervals as in section 8.
6. Compare with the baseline bundle on the same suite version.

6. Online evaluation

Online metrics: [completion rate, user confirmation acceptance rate, handoff rate, refunds or complaints within 7 days, cost and latency]. Link the signals to the [LLM observability specification].

7. Graders

7.1 Programmatic graders

7.2 LLM-as-judge

Calibration: two trained humans label [100] traces. Report agreement between the judge and the human consensus with Cohen's kappa:

where $p_o$ is observed agreement and $p_e$ the agreement expected by chance. Use the judge for gating only when $\kappa$ reaches the level set by [the evaluation lead, Example: 0.7] and recalibrate on every judge or rubric change.

7.3 Human review

• Reviewers label a stratified sample of [10%] of traces each release, including all runs flagged by any grader.
• Reviewers see the rubric, the goal and the trace, not the version.
• Disagreements go to a third reviewer; the result is logged.

8. Statistical method

8.1 Confidence intervals

Report a 95% Wilson interval for each rate. With $x$ successes in $n$ runs, $\hat p = x / n$ and $z = 1.96$:

Because runs of the same task are correlated, also report an interval from a cluster bootstrap that resamples tasks, not runs.

8.2 Sample size

To detect a drop of $\delta$ in a success rate near $p$ with significance $\alpha$ and power $1 - \beta$, the approximate number of independent tasks per version is:

Example: $p = 0.8$, $\delta = 0.05$, $\alpha = 0.05$, power 0.8 gives about $n \approx 1{,}005$ tasks per version. With a smaller suite, widen the regression margin or use paired comparisons on the same tasks.

8.3 Comparing versions

• Use paired comparisons on the same tasks and seeds where possible (McNemar's test for success or failure per task).
• Correct for multiple metrics with [Holm] when several gates are tested together.
• Report effect sizes with intervals, not only p-values.

9. Safety and adversarial cases

ASI05 (code execution), ASI07 (inter-agent communication) and ASI10 (rogue agents): [state whether they apply; Example: "Not applicable, the agent runs no code and has no peer agents; revisit if delegation is added"].

10. Regression gates

A warning needs a written justification from the agent owner in the report; a blocking gate failure stops the release.

11. Reporting

11.1 Summary

[Agent v2] was evaluated on suite [v4] ([200] tasks, [5] runs each) in [ENV-SIM] on YYYY-MM-DD. Example: "Task success rose from 78% to 83% (paired difference 5 points, 95% interval 2 to 8). No safety violations. Cost per task rose 9%. Recommendation: release."

11.2 Results table

11.3 Failure analysis

Group failed runs by cause ([wrong tool, wrong argument, missed confirmation, gave up early, loop, simulator issue]) with counts and two example trace links per group. File each new failure pattern as a task for the next suite version.

12. Cadence and ownership

13. Glossary and references

References:

• OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026, December 2025: https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
• [AI agent specification], [LLM evaluation plan], [Agentic threat model], [AI red team plan and report], [LLM observability specification].

Standard and licence note

Based on common industry practice for evaluating tool-using AI agents: outcome and trajectory metrics, sandboxed environments, calibrated graders and interval-based regression gates. The adversarial case categories reference the OWASP Top 10 for Agentic Applications for 2026 by the OWASP GenAI Security Project (https://genai.owasp.org/), licensed under Creative Commons Attribution-ShareAlike 4.0 (https://creativecommons.org/licenses/by-sa/4.0/); only the category IDs and short names are used, and any adaptation of that OWASP material must be shared under the same licence. The statistical formulas are standard methods. All other text is original; example numbers are illustrative, not recommended thresholds. Check the current OWASP list before use. Template licence: MIT.