Download for MacApple silicon, macOS 14+
Skill

model-evaluation

@awslabs Updated 2026-10-15

Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.

agent-pluginsagent-skillsagentsawscoding-agent-skillscoding-agents

Install

With your agent

Install OMGSkills skill "awslabs/agent-plugins:plugins/sagemaker-ai/skills/model-evaluation" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.

Manual command

git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/model-evaluation ~/.claude/skills/model-evaluation

From README

Model Evaluation Generate code that evaluates a SageMaker model. Prerequisites The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first. Principles One thing at a time. Each response advances exactly one decision. Never combine multiple questions in a single turn. Confirm before proceeding. Wait for the user to agree before moving to the next step. Don't read files until you need them. Only read reference files when you've reached the step that requires them. Don't ask what you already know. If the answer is in conversation history, workflowstate.json, plan.md, or any file you've already read — use it. Confirm if unsure, but don't re-ask. No narration. Share outcomes and ask questions. Keep responses short. No repetition. If you said something before a tool call, don't repeat it after. Scope This skill supports the evaluation feature for SageMaker Serverless Model Customization.