datahub-evals
Use this skill to run DataHub's saved evals and report answers for judging. Triggers on: "run our evals", "run the eval suite", "run eval urn:li:eval:...", "how are our evals doing", "check for eval regressions", "upload this answer as an eval result", "score this answer with the DataHub judge", "compare two agents on the same eval". Answers each eval in a fresh agent with the DataHub tools attached, reports the answer through the DataHub Cloud CLI, and reads back the verdict DataHub's own judge produced.
Install
With your agent
Install OMGSkills skill "datahub-project/datahub-skills:skills/datahub-evals" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/datahub-project/datahub-skills /tmp/datahub-skills && ln -s /tmp/datahub-skills/skills/datahub-evals ~/.claude/skills/datahub-evalsFrom README
DataHub Evals Run DataHub's saved evals, and report answers — yours or another agent's — for DataHub to judge. You are the runner. There is no script: you fetch the evals, answer each one in a fresh agent, and report the answers. Every call to DataHub is one evals subcommand, so the queries and the payload live in the CLI. Use acryl-datahub-cloud evals. A datahub evals form exists in the CLI's own help text and in some notes, but the group is not wired into the datahub CLI in any shipped release — that name answers No such command 'evals', and this skill does not use it. --- Never simulate the judge If DataHub's judge cannot be reached, say so and stop. Do not score the answer yourself, do not ask a subagent to render a verdict "the way DataHub would", and do not present any locally-produced score as a verdict. A simulated verdict written in the house style reads as authoritative, gets pasted into a comparison table, and is comparable with nothing.
More from this repo
datahub-mfe-configure-app
Configure a DataHub instance to load and display a Micro Frontend (MFE) app.
datahub-mfe-create-app
Scaffold a new DataHub Micro Frontend (MFE) app with all boilerplate files.
datahub-sql-workflow
Ground text-to-SQL work in DataHub catalog evidence.