# spark-engineer

> Claude/Codex skill by [@jeffallan](https://omgskills.com/library/jeffallan/index.md) · 12k stars · Updated 2026-10-30

Use when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads. Invoke to write DataFrame transformations, optimize Spark SQL queries, implement RDD pipelines, tune shuffle operations, configure executor memory, process .parquet files, handle data partitioning, or build structured streaming analytics.

Tags: ai-agents, claude, claude-code, claude-marketplace, claude-skills

## Install

```sh
git clone https://github.com/Jeffallan/claude-skills /tmp/claude-skills && ln -s /tmp/claude-skills/skills/spark-engineer ~/.claude/skills/spark-engineer
```

## From README

Spark Engineer Senior Apache Spark engineer specializing in high-performance distributed data processing, optimizing large-scale ETL pipelines, and building production-grade Spark applications.

## Links

- HTML page: https://omgskills.com/library/jeffallan/claude-skills/spark-engineer/
- GitHub: https://github.com/Jeffallan/claude-skills
- Author: [@jeffallan](https://omgskills.com/library/jeffallan/index.md)

## More from this repo

- [api-designer](https://omgskills.com/library/jeffallan/claude-skills/api-designer/index.md) — Used when designing REST or GraphQL APIs, creating OpenAPI specifications, or planning API architecture.
   - [@jeffallan](https://omgskills.com/library/jeffallan/index.md) · 12k stars
- [architecture-designer](https://omgskills.com/library/jeffallan/claude-skills/architecture-designer/index.md) — Used when designing new high-level system architecture, reviewing existing designs, or making architectural decisions.
   - [@jeffallan](https://omgskills.com/library/jeffallan/index.md) · 12k stars
- [atlassian-mcp](https://omgskills.com/library/jeffallan/claude-skills/atlassian-mcp/index.md) — Integrates with Atlassian products to manage project tracking and documentation via MCP protocol.
   - [@jeffallan](https://omgskills.com/library/jeffallan/index.md) · 12k stars