tavily-crawl
Crawl websites and extract content from multiple pages via the Tavily CLI. Use this skill when the user wants to crawl a site, download documentation, extract an entire docs section, bulk-extract pages, save a site as local markdown files, or says "crawl", "get all the pages", "download the docs", "extract everything under /docs", "bulk extract", or needs content from many pages on the same domain. Supports depth/breadth control, path filtering, semantic instructions, and saving each page as a local markdown file.
Install
With your agent
Install OMGSkills skill "tavily-ai/skills:tavily-crawl" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/tavily-ai/skills /tmp/skills && ln -s /tmp/skills/skills/tavily-crawl ~/.claude/skills/tavily-crawlFrom README
tavily crawl Crawl a website and extract content from multiple pages. Supports saving each page as a local markdown file. Before running Crawl requires authentication. Run the requested command directly when tvly is already authenticated; do not add a status check to every invocation. If tvly is missing, follow the tavily-cli . If an installed CLI reports an authentication error, use tvly login for authentication only, or tvly init --skip-skills when guided verification is also useful. Browser-based OAuth is preferred when an interactive user can complete it. --no-browser prints the sign-in link instead of opening it, but still waits for a localhost callback. In an unattended agent or CI environment, leave authentication to the user or use a securely provided TAVILYAPIKEY. Do not start a second login immediately after guided has completed.
More from this repo
tavily-best-practices
Used to build production-ready Tavily integrations with best practices baked in.
tavily-cli
Set up, authenticate, update, troubleshoot, or choose between Tavily CLI web commands.
tavily-dynamic-search
Programmatic Tavily search with context isolation.