# output-eval-error-analysis

> Claude/Codex skill by [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars · Updated 2026-10-22

Systematically review workflow traces to identify failure modes before building evaluators. Use when starting an eval project, after significant pipeline changes, or when production quality drops.

Tags: agents, ai, ai-agents, ai-sdk, claude, claude-code, javascript, llm, nodejs, open-source, reliability, temporal, typescript, workflow-engine, workflows

## Install

```sh
git clone https://github.com/growthxai/output /tmp/output && ln -s /tmp/output/coding_assistants/claude/plugins/outputai/skills/output-eval-error-analysis ~/.claude/skills/output-eval-error-analysis
```

## From README

Error Analysis for Workflow Evaluation Overview Review real workflow traces and categorize how your workflow fails before writing any evaluators. Evaluators built without error analysis target generic qualities ("is this good?") instead of the specific ways your workflow actually breaks. This skill walks you through the process. When to Use Starting a new eval project for an existing workflow Production quality has dropped and you need to understand why After significant prompt, model, or pipeline changes Before building your first evaluator for a workflow Step 1: Collect Traces Gather 50-100 representative workflow executions. More traces = more reliable failure categories. From recent runs List recent workflow executions and pull their traces: From production (bulk download) Download production traces directly into dataset YAML files: This creates YAML files in tests/datasets/ with the input and lastoutput fields populated from real executions.

## Links

- HTML page: https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-eval-error-analysis/
- GitHub: https://github.com/growthxai/output
- Author: [@growthxai](https://omgskills.com/library/growthxai/index.md)

## More from this repo

- [output-build-workflow](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-build-workflow/index.md) — Implement an Output SDK workflow from a plan document.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars
- [output-credentials-edit](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-credentials-edit/index.md) — View, edit, and set encrypted credentials in an Output.ai project.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars
- [output-credentials-env-vars](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-credentials-env-vars/index.md) — Wire encrypted credentials to environment variables using the credential: convention.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars