# hyperpod-slurm-debugger

> Claude/Codex skill by [@awslabs](https://omgskills.com/library/awslabs/index.md) · 927 stars · Updated 2026-10-15

Diagnostic-only skill for Slurm scheduler and node-daemon issues on Amazon SageMaker HyperPod Slurm clusters. Scope mirrors the HyperPod troubleshooting guide. Invoke when the user reports a Slurm node stuck in down/drain, "Node unexpectedly rebooted" after auto-repair, slurmd not running, jobs stuck PENDING with REASON=Resources while sinfo shows idle nodes, jobs stuck COMPLETING after node replacement, GRES/GPU counts wrong, scontrol ping failing, slurmctld unresponsive, an Action:Reboot/Replace request that did not trigger HyperPod auto-recovery, or auto-resume not restarting a job. Also triggers on "drain before reboot", "diagnose a Slurm node", "investigate stuck jobs."

Tags: agent-plugins, agent-skills, agents, aws, coding-agent-skills, coding-agents

## Install

```sh
git clone https://github.com/awslabs/agent-plugins /tmp/agent-plugins && ln -s /tmp/agent-plugins/plugins/sagemaker-ai/skills/hyperpod-slurm-debugger ~/.claude/skills/hyperpod-slurm-debugger
```

## From README

HyperPod Slurm Debugger Diagnostic-only. Identify and classify Slurm scheduler and node-daemon issues on HyperPod Slurm clusters. Do not run, recommend, or print any state-mutating command. For remediation, link to the official AWS or Slurm documentation. When to invoke Invoke when the user reports any of the symptoms in the decision table. When NOT to invoke Cluster has Orchestrator.Eks — invoke hyperpod-node-debugger or hyperpod-nccl. Single-node hardware fault with healthy Slurm scheduler — invoke hyperpod-node-debugger. NCCL training-hang investigation — invoke hyperpod-nccl. Node unreachable via SSM — invoke hyperpod-ssm. Constraints Read-only. Do not run, recommend, or print state-mutating commands. For any remediation, link to AWS or Slurm docs. The user authorizes and executes. IaC-managed cluster (Terraform / CloudFormation / CDK): warn that direct mutation drifts the live state from the IaC plan.

## Links

- HTML page: https://omgskills.com/library/awslabs/agent-plugins/plugins/sagemaker-ai/skills/hyperpod-slurm-debugger/
- GitHub: https://github.com/awslabs/agent-plugins
- Author: [@awslabs](https://omgskills.com/library/awslabs/index.md)

## More from this repo

- [amplify-workflow](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-amplify/skills/amplify-workflow/index.md) — Used to build and deploy full-stack web and mobile apps with AWS Amplify Gen2 (TypeScript code-first).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [api-gateway](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/api-gateway/index.md) — Build, manage, and operate APIs with Amazon API Gateway (REST, HTTP, and WebSocket).
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars
- [aws-lambda](https://omgskills.com/library/awslabs/agent-plugins/plugins/aws-serverless/skills/aws-lambda/index.md) — Design, build, deploy, test, and debug serverless applications with AWS Lambda.
   - [@awslabs](https://omgskills.com/library/awslabs/index.md) · 933 stars