# spark-engineer

> Claude/Codex skill by [@jeffallan](https://omgskills.com/library/jeffallan/index.md) · 8.8k stars · Updated 2026-05-01

Use when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads. Invoke to write DataFrame transformations, optimize Spark SQL queries, implement RDD pipelines, tune shuffle operations, configure executor memory, process .parquet files, handle data partitioning, or build structured streaming analytics.

Tags: ai-agents, claude, claude-code, claude-marketplace, claude-skills

## Install

```sh
git clone https://github.com/Jeffallan/claude-skills /tmp/claude-skills && ln -s /tmp/claude-skills/skills/spark-engineer ~/.claude/skills/spark-engineer
```

## Links

- HTML page: https://omgskills.com/skills/jeffallan/claude-skills/spark-engineer/
- GitHub: https://github.com/Jeffallan/claude-skills
- Author: [@jeffallan](https://omgskills.com/library/jeffallan/index.md)

## More from this repo

- [django-storages-s3](https://omgskills.com/skills/jeffallan/claude-skills/skills/django-storages-s3/index.md) — Used when configuring Django to store static and media files on AWS S3 with django-storages.
   - [@Jeffallan](https://omgskills.com/library/jeffallan/index.md) · 12k stars
- [rag-architect](https://omgskills.com/skills/jeffallan/claude-skills/skills/rag-architect/index.md) — Designs and implements production-grade RAG systems by chunking documents, generating embeddings, configuring vector...
   - [@Jeffallan](https://omgskills.com/library/jeffallan/index.md) · 11k stars
- [terraform-engineer](https://omgskills.com/skills/jeffallan/claude-skills/skills/terraform-engineer/index.md) — Used when implementing infrastructure as code with Terraform across AWS, Azure, or GCP.
   - [@Jeffallan](https://omgskills.com/library/jeffallan/index.md) · 11k stars