# dataset-evaluation

> Claude/Codex skill by [@awslabs](https://omgskills.com/library/awslabs/index.md) · 927 stars · Updated 2026-10-15

Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR). Use when the user says "is my dataset okay", "evaluate my data", "check my training data", "I have my own data", or before starting any fine-tuning job. Detects file format, checks schema compliance against the selected model and technique, and reports whether the data is ready for training or evaluation.

Tags: agent-plugins, agent-skills, agents, aws, coding-agent-skills, coding-agents

## Install

```sh
git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/dataset-evaluation ~/.claude/skills/dataset-evaluation
```

## From README

Workflow Instruction Follow the workflow shown below. Locate the dataset, check the file type, and resolve any issues with missing files or wrong file types. Determine the fine-tuning model and fine-tuning strategy. Run the appropriate validation based on the model family. Summarize the results: is the dataset ready for fine-tuning? Prerequisites The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first. --- Workflow Locate Dataset: The full path may be a local file path, or an S3 URI Resolve the full path to the dataset file, make sure read permissions are available, and help the user if the file is not found Determine strategy and model: File formatting depends on the currently selected fine-tuning strategy and fine-tuning base model. If the strategy and model are already known from the conversation context (e.g., selected via the model-selection and finetuning-technique skills), use them.

## Links

- HTML page: https://omgskills.com/library/awslabs/agent-plugins/plugins/sagemaker-ai/skills/dataset-evaluation/
- GitHub: https://github.com/awslabs/agent-plugins
- Author: [@awslabs](https://omgskills.com/library/awslabs/index.md)

## More from this repo

- [amplify-workflow](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-amplify/skills/amplify-workflow/index.md) — Used to build and deploy full-stack web and mobile apps with AWS Amplify Gen2 (TypeScript code-first).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [api-gateway](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/api-gateway/index.md) — Build, manage, and operate APIs with Amazon API Gateway (REST, HTTP, and WebSocket).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [aws-lambda](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/aws-lambda/index.md) — Design, build, deploy, test, and debug serverless applications with AWS Lambda.
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars