# agent-evaluation

> Claude/Codex skill by [@mlflow](https://omgskills.com/library/mlflow/index.md) · 82 stars · Updated 2026-10-23

Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).

## Install

```sh
git clone https://github.com/mlflow/skills /tmp/skills && ln -s /tmp/skills/agent-evaluation ~/.claude/skills/agent-evaluation
```

## From README

Agent Evaluation with MLflow Comprehensive guide for evaluating GenAI agents with MLflow. Use this skill for the complete evaluation workflow or individual components - tracing , environment configuration, dataset creation, scorer definition, or evaluation execution. Each section can be used independently based on your needs. ⛔ CRITICAL: Must Use MLflow APIs DO NOT create custom evaluation frameworks. You MUST use MLflow's native APIs: Datasets: Use mlflow.genai.datasets.createdataset() - NOT custom test case files Scorers: Use MLflow built-ins, @scorer, or mlflow.genai.judges.makejudge() - NOT a custom evaluation framework Evaluation: Use mlflow.genai.evaluate() - NOT custom evaluation loops Scripts: Use the provided scripts/ directory templates - NOT custom evaluation/ directories Why? MLflow tracks everything (datasets, scorers, traces, results) in the experiment. Custom frameworks bypass this and lose all observability.

## Links

- HTML page: https://omgskills.com/library/mlflow/skills/agent-evaluation/
- GitHub: https://github.com/mlflow/skills
- Author: [@mlflow](https://omgskills.com/library/mlflow/index.md)

## More from this repo

- [analyzing-mlflow-session](https://omgskills.com/library/mlflow/skills/analyze-mlflow-chat-session/index.md) — Analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interaction.
   - [@mlflow](https://omgskills.com/library/mlflow/index.md) · 82 stars
- [analyzing-mlflow-trace](https://omgskills.com/library/mlflow/skills/analyze-mlflow-trace/index.md) — Analyzes a single MLflow trace to answer a user query about it.
   - [@mlflow](https://omgskills.com/library/mlflow/index.md) · 82 stars
- [build-a-scorer](https://omgskills.com/library/mlflow/skills/build-a-scorer/index.md) — Help the user choose, create, iterate, or align MLflow judges and scorers for an agent, RAG app, LLM app, or GenAI...
   - [@mlflow](https://omgskills.com/library/mlflow/index.md) · 82 stars