dataset-evaluation
Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR). Use when the user says "is my dataset okay", "evaluate my data", "check my training data", "I have my own data", or before starting any fine-tuning job. Detects file format, checks schema compliance against the selected model and technique, and reports whether the data is ready for training or evaluation.
Install
With your agent
Install OMGSkills skill "awslabs/agent-plugins:plugins/sagemaker-ai/skills/dataset-evaluation" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/dataset-evaluation ~/.claude/skills/dataset-evaluationFrom README
Workflow Instruction Follow the workflow shown below. Locate the dataset, check the file type, and resolve any issues with missing files or wrong file types. Determine the fine-tuning model and fine-tuning strategy. Run the appropriate validation based on the model family. Summarize the results: is the dataset ready for fine-tuning? Prerequisites The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first. --- Workflow Locate Dataset: The full path may be a local file path, or an S3 URI Resolve the full path to the dataset file, make sure read permissions are available, and help the user if the file is not found Determine strategy and model: File formatting depends on the currently selected fine-tuning strategy and fine-tuning base model. If the strategy and model are already known from the conversation context (e.g., selected via the model-selection and finetuning-technique skills), use them.
More from this repo
amplify-workflow
Used to build and deploy full-stack web and mobile apps with AWS Amplify Gen2 (TypeScript code-first).
api-gateway
Build, manage, and operate APIs with Amazon API Gateway (REST, HTTP, and WebSocket).
aws-lambda
Design, build, deploy, test, and debug serverless applications with AWS Lambda.