# spark-lake-catalog

> Claude/Codex skill by [@oleanderhq](https://omgskills.com/library/oleanderhq/index.md) · 4 stars · Updated 2026-08-10

Spark patterns for reading and writing oleander lake catalog tables: spark.table(), append vs overwrite, and avoiding driver-side writes. Use when building Spark jobs that read or write Iceberg tables in the oleander catalog.

## Install

```sh
git clone https://github.com/OleanderHQ/claude-plugin /tmp/claude-plugin && ln -s /tmp/claude-plugin/skills/spark-lake-catalog ~/.claude/skills/spark-lake-catalog
```

## From README

oleander Spark Lake Catalog Use this skill when reading from or writing to the oleander lake catalog in a Spark job. For shared catalog conventions such as table naming, namespaces, and avoiding raw storage paths, also use lake-catalog. Reading tables Use spark.table() with the fully qualified table name: Do not construct raw storage paths for Iceberg tables. Use catalog-qualified names so Spark reads the table through the Iceberg catalog. Writing tables Append (add rows to an existing or new table): Overwrite (replace table contents): Use writeTo(...).append() for incremental pipelines. Use write.mode("overwrite").saveAsTable(...) when replacing the full result set each run. Prefer Spark writes over driver writes Avoid collecting data to the driver and then writing from Python memory. Keep writes as Spark DataFrame operations so Iceberg handles the transaction, partitioning, and metadata correctly.

## Links

- HTML page: https://omgskills.com/library/oleanderhq/claude-plugin/skills/spark-lake-catalog/
- GitHub: https://github.com/OleanderHQ/claude-plugin
- Author: [@oleanderhq](https://omgskills.com/library/oleanderhq/index.md)

## More from this repo

- [lake-catalog](https://omgskills.com/library/oleanderhq/claude-plugin/skills/lake-catalog/index.md) — Engine-agnostic oleander lake catalog conventions: catalog.namespace.table naming, hierarchy, and catalog-qualified...
   - [@oleanderhq](https://omgskills.com/library/oleanderhq/index.md) · 4 stars
- [lake-query](https://omgskills.com/library/oleanderhq/claude-plugin/skills/lake-query/index.md) — Runs lake SQL through oleander's query router: query\_run for reads, query\_submit for writes, spark\_sql\_submit for named...
   - [@oleanderhq](https://omgskills.com/library/oleanderhq/index.md) · 4 stars
- [polars-submit](https://omgskills.com/library/oleanderhq/claude-plugin/skills/polars-submit/index.md) — Runs Polars queries or scripts on oleander via the CLI, in local or distributed mode, and saves results to the lake...
   - [@oleanderhq](https://omgskills.com/library/oleanderhq/index.md) · 4 stars