# prometheus-cardinality-troubleshooter

> Claude/Codex skill by [@grafana](https://omgskills.com/library/grafana/index.md) · 286 stars · Updated 2026-10-17

Diagnostic guide for active Prometheus cardinality problems — slow queries, OOMing Prometheus, high Grafana Cloud Active Series or DPM bills, "too many samples" ingest errors, series churn, or rapid memory growth. Walks through tsdb status endpoints, per-metric and per-label drill-downs, common-culprit galleries, and remediation paths. Use when the user is \*currently experiencing\* a cardinality fire. For preventing cardinality issues at the source, route to prometheus-label-strategy. For post-ingest aggregation, route to adaptive-metrics. For DPM-specific analysis, route to dpm-finder.

## Install

```sh
git clone https://github.com/grafana/skills /tmp/skills && ln -s /tmp/skills/skills/grafana-cloud/prometheus-cardinality-troubleshooter ~/.claude/skills/prometheus-cardinality-troubleshooter
```

## From README

Prometheus Cardinality Troubleshooter You are an expert in diagnosing live Prometheus cardinality problems. When a user reports a Prometheus performance, memory, or cost issue that smells like cardinality, use this guide to triage systematically. This skill is diagnostic and operational. For schema design and prevention, route to prometheus-label-strategy. --- Before You Remediate: The One Rule Under pressure, the tempting move is to labeldrop the high-cardinality label at scrape time. Do not. You cannot remove, at scrape time, any label that makes a series unique — not pod, not instance, not anything that distinguishes one real series from another. It looks like it stops the bleeding; it actually breaks the data: Counter resets from different series get merged → rate() and increase() return garbage, often absurdly high values. Multiple samples land on the same series per scrape → duplicate-sample / out-of-order errors and inflated DPM, not reduced.

## Links

- HTML page: https://omgskills.com/library/grafana/skills/prometheus-cardinality-troubleshooter/
- GitHub: https://github.com/grafana/skills
- Author: [@grafana](https://omgskills.com/library/grafana/index.md)

## More from this repo

- [k6-docs](https://omgskills.com/library/grafana/skills/k6-docs/index.md) — Used to write or review k6 documentation across the three k6 repositories - k6-DefinitelyTyped (TypeScript types),...
   - [@grafana](https://omgskills.com/library/grafana/index.md) · 295 stars
- [adaptive-metrics](https://omgskills.com/library/grafana/skills/adaptive-metrics/index.md) — Cut Grafana Cloud Metrics cost by shrinking active-series count with Adaptive Metrics aggregation rules —...
   - [@grafana](https://omgskills.com/library/grafana/index.md) · 286 stars
- [admin](https://omgskills.com/library/grafana/skills/admin/index.md) — Used to manage Grafana Cloud accounts — organizations, stacks, RBAC roles and assignments, SSO/SAML/OAuth/GitHub auth,...
   - [@grafana](https://omgskills.com/library/grafana/index.md) · 286 stars