# model-evaluation

> Claude/Codex skill by [@awslabs](https://omgskills.com/library/awslabs/index.md) · 927 stars · Updated 2026-10-15

Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.

Tags: agent-plugins, agent-skills, agents, aws, coding-agent-skills, coding-agents

## Install

```sh
git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/model-evaluation ~/.claude/skills/model-evaluation
```

## From README

Model Evaluation Generate code that evaluates a SageMaker model. Prerequisites The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first. Principles One thing at a time. Each response advances exactly one decision. Never combine multiple questions in a single turn. Confirm before proceeding. Wait for the user to agree before moving to the next step. Don't read files until you need them. Only read reference files when you've reached the step that requires them. Don't ask what you already know. If the answer is in conversation history, workflowstate.json, plan.md, or any file you've already read — use it. Confirm if unsure, but don't re-ask. No narration. Share outcomes and ask questions. Keep responses short. No repetition. If you said something before a tool call, don't repeat it after. Scope This skill supports the evaluation feature for SageMaker Serverless Model Customization.

## Links

- HTML page: https://omgskills.com/library/awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-evaluation/
- GitHub: https://github.com/awslabs/agent-plugins
- Author: [@awslabs](https://omgskills.com/library/awslabs/index.md)

## More from this repo

- [amplify-workflow](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-amplify/skills/amplify-workflow/index.md) — Used to build and deploy full-stack web and mobile apps with AWS Amplify Gen2 (TypeScript code-first).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [api-gateway](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/api-gateway/index.md) — Build, manage, and operate APIs with Amazon API Gateway (REST, HTTP, and WebSocket).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [aws-lambda](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/aws-lambda/index.md) — Design, build, deploy, test, and debug serverless applications with AWS Lambda.
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars