federate-lakehouse-catalog
Sets up Google Cloud Lakehouse federated catalogs to remote Iceberg REST Catalogs. Currently supported catalogs: Databricks Unity, AWS Glue. Supported clouds hosting those catalogs: GCP, AWS. The primary use case is connecting to remote data to query it from GCP engines (BigQuery, Spark). Examples of when to use this: "federate my lakehouse catalog to databricks", "query data in databricks", "query data in s3", "connect to aws glue". Do NOT use for direct remote database SQL execution (e.g., Databricks SQL) or managing remote clusters and infrastructure (e.g., Databricks clusters, AWS Glue jobs).
Install
With your agent
Install OMGSkills skill "googlecloudplatform/data-agent-kit-plugin:skills/federate-lakehouse-catalog" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/GoogleCloudPlatform/data-agent-kit-plugin /tmp/data-agent-kit-plugin && ln -s /tmp/data-agent-kit-plugin/skills/federate-lakehouse-catalog ~/.claude/skills/federate-lakehouse-catalogFrom README
Federate Lakehouse Catalog via Cross-cloud Lakehouse This skill describes how to set up a federated catalog in BigQuery to query remote catalogs like Databricks Unity Catalog or AWS Glue Data Catalog data in AWS over the public internet. Prerequisites For Databricks: Databricks Workspace URL and OAuth Service Principal (Client ID and Secret) with read access. For AWS Glue: AWS Administrator access to create IAM roles and permissions policies. Active Google Cloud project with administrative access to create lakehouse resources, and secrets in the case of Databricks. Procedure Step 1: Information Gathering and Region Selection Before running any commands, the agent MUST collect the following information from the user: Determine which catalog the user wants to federate to (e.g., Databricks Unity or AWS Glue) and verify it is supported. Determine where the remote data is located (the specific AWS region).
More from this repo
accidental-data-loss-prevention
**STOP AND VERIFY**: Before running any command or tool that results in irreversible data loss, you MUST obtain...
bigquery-ai-ml
Leverages BigQuery's built-in machine learning and GenAI capabilities for advanced data analytics.
bigquery-bigframes
Generates Python code using BigQuery DataFrames (BigFrames).