airflow-dag-patterns
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
Install
git clone https://github.com/wshobson/agents /tmp/agents && ln -s /tmp/agents/plugins/data-engineering/skills/airflow-dag-patterns ~/.claude/skills/airflow-dag-patterns
From README
Apache Airflow DAG Patterns Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies. When to Use This Skill Creating data pipeline orchestration with Airflow Designing DAG structures and dependencies Implementing custom operators and sensors Testing Airflow DAGs locally Setting up Airflow in production Debugging failed DAG runs Core Concepts DAG Design Principles | Principle | Description | | --------------- | ----------------------------------- | | Idempotent | Running twice produces same result | | Atomic | Tasks succeed or fail completely | | Incremental | Process only new/changed data | | Observable | Logs, metrics, alerts at every step | Task Dependencies Detailed patterns and worked examples Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.
More from this repo
python-packaging
Used to create distributable Python packages with proper project structure, setup.py/pyproject.toml, and publishing to...
ai-debt-detector
Use after generating code, after accepting AI suggestions, or when reviewing AI-written modules.
market-sizing-analysis
Calculate TAM/SAM/SOM for market opportunities using top-down, bottom-up, and value theory methodologies.
