# output-eval-dataset-design

> Claude/Codex skill by [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars · Updated 2026-10-22

Design diverse eval datasets using dimension-based variation. Use when bootstrapping eval datasets, when real traces are sparse, or when existing datasets miss edge cases.

Tags: agents, ai, ai-agents, ai-sdk, claude, claude-code, javascript, llm, nodejs, open-source, reliability, temporal, typescript, workflow-engine, workflows

## Install

```sh
git clone https://github.com/growthxai/output /tmp/output && ln -s /tmp/output/coding_assistants/claude/plugins/outputai/skills/output-eval-dataset-design ~/.claude/skills/output-eval-dataset-design
```

## From README

Designing Eval Datasets Overview Diverse datasets catch more failures. This skill teaches dimension-based dataset design — systematically varying inputs along axes that target failure-prone regions of your workflow. The output is a set of YAML dataset files ready for output workflow test. When to Use Bootstrapping an eval dataset for a new workflow Existing datasets only cover happy paths Real production traces are sparse (fewer than 50) Stress-testing specific failure hypotheses from error analysis When NOT to Use You already have 100+ representative real traces — use stratified sampling from those instead of generating synthetic data You haven't done error analysis yet — do that first (output-eval-error-analysis) so your dimensions target real failure modes, not guesses Step 1: Define Dimensions Identify 3+ axes of input variation that target anticipated failure modes. Each dimension should vary one aspect of the input that you expect to influence output quality.

## Links

- HTML page: https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-eval-dataset-design/
- GitHub: https://github.com/growthxai/output
- Author: [@growthxai](https://omgskills.com/library/growthxai/index.md)

## More from this repo

- [output-build-workflow](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-build-workflow/index.md) — Implement an Output SDK workflow from a plan document.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars
- [output-credentials-edit](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-credentials-edit/index.md) — View, edit, and set encrypted credentials in an Output.ai project.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars
- [output-credentials-env-vars](https://omgskills.com/library/growthxai/output/coding_assistants/claude/plugins/outputai/skills/output-credentials-env-vars/index.md) — Wire encrypted credentials to environment variables using the credential: convention.
   - [@growthxai](https://omgskills.com/library/growthxai/index.md) · 443 stars