dataset-evaluation
Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR). Use when the user says "is my dataset okay", "evaluate my data", "check my training data", "I have my own data", or before starting any fine-tuning job. Detects file format, checks schema compliance against the selected model and technique, and reports whether the data is ready for training or evaluation.
Install
With your agent
Install OMGSkills skill "awslabs/agent-plugins:plugins/sagemaker-ai/skills/dataset-evaluation" into this agent's normal skills directory: use the OMGSkills MCP get_skill tool, install only the pinned_install commit and path, verify SKILL.md against pinned_install.skill_md_sha, and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/dataset-evaluation ~/.claude/skills/dataset-evaluationFrom README
Workflow Instruction Follow the workflow shown below. Locate the dataset, check the file type, and resolve any issues with missing files or wrong file types. Determine the fine-tuning model and fine-tuning strategy. Run the appropriate validation based on the model family. Summarize the results: is the dataset ready for fine-tuning? Prerequisites The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first. --- Workflow Locate Dataset: The full path may be a local file path, or an S3 URI Resolve the full path to the dataset file, make sure read permissions are available, and help the user if the file is not found Determine strategy and model: File formatting depends on the currently selected fine-tuning strategy and fine-tuning base model. If the strategy and model are already known from the conversation context (e.g., selected via the model-selection and finetuning-technique skills), use them.
More from this repo
aws-lambda-microvms
Build, run, debug, and operate applications on AWS Lambda MicroVMs — Firecracker-isolated, snapshot-resumable...
directory-management
Manages project directory setup and artifact organization.
finetuning
Generates code that fine-tunes a base model using SageMaker serverless training jobs.