arize-experiment
Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment, compare models, model performance, evaluate AI, experiment results, benchmark, A/B test models, or measure accuracy.
Install
With your agent
Install OMGSkills skill "arize-ai/arize-skills:arize-experiment" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/Arize-ai/arize-skills /tmp/arize-skills && ln -s /tmp/arize-skills/skills/arize-experiment ~/.claude/skills/arize-experiment
From README
Arize Experiment Skill SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list. Concepts Experiment = a named evaluation run against a specific dataset version, containing one run per example Experiment Run = the result of processing one dataset example -- includes the model output, optional evaluations, and optional metadata Dataset = a versioned collection of examples; every experiment is tied to a dataset and a specific dataset version Evaluation = a named metric attached to a run (e.g., correctness, relevance), with optional label, score, and explanation The typical flow: export a dataset → process each example → collect outputs and evaluations → create an experiment with the runs. Prerequisites Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront.
More from this repo
arize-instrumentation
Adds Arize AX tracing to an LLM application for the first time.
arize-admin
Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the...
arize-ai-provider-integration
Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and...
