arize-evaluator
Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, code evaluator, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.
Install
With your agent
Install OMGSkills skill "arize-ai/arize-skills:arize-evaluator" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/Arize-ai/arize-skills /tmp/arize-skills && ln -s /tmp/arize-skills/skills/arize-evaluator ~/.claude/skills/arize-evaluatorFrom README
Arize Evaluator Skill SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list. This skill covers designing, creating, and running evaluators on Arize — both LLM-as-judge (template) evaluators and code evaluators (deterministic, no LLM required). An evaluator defines the judge; a task is how you run it against real data. --- Prerequisites Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront. If an ax command fails, troubleshoot based on the error: command not found or version error → see references/ax- .md 401 Unauthorized / missing API key → run ax profiles show to inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it.
More from this repo
arize-instrumentation
Adds Arize AX tracing to an LLM application for the first time.
arize-admin
Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the...
arize-ai-provider-integration
Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and...