---
name: agent-workflow-evaluation
description: "Design or interpret a bounded evaluation of an engineering agent workflow using representative tasks, quality criteria, full costs, and uncertainty. Use for pilot decisions and evaluation reports, not unsupported ROI claims."
metadata:
  author: Vibe Haus
  version: "1.0"
---

# Agent workflow evaluation

## Inputs
Identify the adoption decision, task family, current baseline, candidate workflow, available data, quality requirements, budget, and observation window. Distinguish planning an experiment from analyzing completed observations.

## Procedure
- Define the unit of work and acceptance standard before selecting metrics. Keep failed and abandoned attempts in the denominator.
- Select representative task categories and reproducible starting states. Prevent exposure of historical solutions where feasible; document possible contamination.
- Separate development tasks used for tuning from held-out tasks used for the decision.
- Choose a comparison that answers the decision. Record assignment method, model and tool versions, context, attempt limits, and deviations. Account for learning when people repeat tasks.
- Measure accepted outcomes, quality failures, human framing/review/rework effort, model/tool/infrastructure cost, and lead time. Avoid double-counting concurrent human or agent time.
- Predeclare quality and cost gates when planning. When analyzing existing data, disclose retrospectively chosen thresholds instead of pretending they were predeclared.
- Report results by relevant task category with sample sizes, missing observations, and uncertainty. Do not infer causation from uncontrolled before/after comparisons.

## Output
For a plan: decision, task set, baseline, candidate, acceptance rubric, measurement procedure, budgets, stopping conditions, and limitations.
For a report: observed results, denominators, total and per-accepted-outcome costs, failures, exclusions, uncertainty, and a scoped adopt / continue / revise / stop recommendation. Label examples as illustrative and distinguish measured facts from hypotheses.

## Boundaries
Do not fabricate trials, benchmarks, costs, approvals, or customer results. Do not execute paid or consequential experiments without the user's applicable authorization. If essential data is absent, provide a bounded plan or explain which conclusion remains unsupported.

## Reuse

You may use and adapt this original Vibe Haus template with attribution. Review it before installation and adapt it to your repository's actual policies.