# phoenix-evals

> Claude/Codex skill by [@arize-ai](https://omgskills.com/library/arize-ai/index.md) · 12k stars · Updated 2026-10-17

Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and validate judges against human-labeled data.

Tags: agents, ai-monitoring, ai-observability, aiengineering, anthropic, datasets, evals, langchain, llamaindex, llm-eval, llm-evaluation, llmops, llms, openai, prompt-engineering, smolagents

## Install

```sh
git clone https://github.com/Arize-ai/phoenix /tmp/phoenix && ln -s /tmp/phoenix/.agents/skills/phoenix-evals ~/.claude/skills/phoenix-evals
```

## From README

Phoenix Evals Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans. Quick Reference | Task | Files | | ---- | ----- | | | -python, -typescript | | Decide what to evaluate | evaluators-overview | | Choose a judge model | fundamentals-model-selection | | Use pre-built evaluators | evaluators-pre-built | | Build code evaluator | evaluators-code-python, evaluators-code-typescript | | Build LLM evaluator | evaluators-llm-python, evaluators-llm-typescript, evaluators-custom-templates | | Batch evaluate DataFrame | evaluate-dataframe-python | | Run experiment | experiments-running-python, experiments-running-typescript | | Run evals in a test runner (CI gate) | integrations-pytest, integrations-vitest-jest | | Create dataset | experiments-datasets-python, experiments-datasets-typescript | | Generate synthetic data | experiments-synthetic-python, experiments-synthetic-typescript | | Validate evaluator accuracy | validation...

## Links

- HTML page: https://omgskills.com/skills/arize-ai/phoenix/phoenix-evals/
- GitHub: https://github.com/Arize-ai/phoenix
- Author: [@arize-ai](https://omgskills.com/library/arize-ai/index.md)

## More from this repo

- [phoenix-cli](https://omgskills.com/skills/arize-ai/phoenix/phoenix-cli/index.md) — Used to debug LLM applications using the Phoenix CLI (\`px\`, from @arizeai/phoenix-cli).
   - [@arize-ai](https://omgskills.com/library/arize-ai/index.md) · 12k stars
- [phoenix-tracing](https://omgskills.com/skills/arize-ai/phoenix/phoenix-tracing/index.md) — OpenInference semantic conventions and instrumentation for Phoenix AI observability.
   - [@arize-ai](https://omgskills.com/library/arize-ai/index.md) · 12k stars

## More skills by this author

- [arize-admin](https://omgskills.com/skills/arize-ai/arize-skills/arize-admin/index.md) — Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the...
   - [@arize-ai](https://omgskills.com/library/arize-ai/index.md) · 65 stars