# diffbot-crawl

> Claude/Codex skill by [@diffbot](https://omgskills.com/library/diffbot/index.md) · 5 stars · Updated 2026-09-28

Crawl a website or manage Diffbot crawler jobs. Use when the user wants to crawl a site for structured content, list existing crawl jobs, or delete a crawl job. Triggers on: crawl website, crawl site, crawler job, list crawl jobs, delete crawl job, diffbot crawl.

Tags: agent-skills, ai, coding, crawler, entity-resolution, extract, extraction, knowledge, knowledge-graph, knowledge-graphs, research, skills, websearch, websearch-agent, websearch-api

## Install

```sh
git clone https://github.com/diffbot/diffbot-skills /tmp/diffbot-skills && ln -s /tmp/diffbot-skills/skills/diffbot-crawl ~/.claude/skills/diffbot-crawl
```

## From README

Diffbot Crawler Crawl websites for structured content via the Diffbot Crawlbot API, and manage crawler jobs. The db crawl CLI commands Always invoke with the fixed path /.diffbot/venv/bin/db.

## Links

- HTML page: https://omgskills.com/library/diffbot/diffbot-skills/skills/diffbot-crawl/
- GitHub: https://github.com/diffbot/diffbot-skills
- Author: [@diffbot](https://omgskills.com/library/diffbot/index.md)

## More from this repo

- [diffbot-deals](https://omgskills.com/library/diffbot/diffbot-skills/skills/diffbot-deals/index.md) — Used to search funding rounds, investments, and acquisitions in the Diffbot Knowledge Graph by date, deal size, round...
   - [@diffbot](https://omgskills.com/library/diffbot/index.md) · 5 stars
- [diffbot-dql](https://omgskills.com/library/diffbot/diffbot-skills/skills/diffbot-dql/index.md) — Query the Diffbot Knowledge Graph directly with DQL (Diffbot Query Language).
   - [@diffbot](https://omgskills.com/library/diffbot/index.md) · 5 stars
- [diffbot-entities](https://omgskills.com/library/diffbot/diffbot-skills/skills/diffbot-entities/index.md) — Identify and resolve named entities in text using the Diffbot NLP API.
   - [@diffbot](https://omgskills.com/library/diffbot/index.md) · 5 stars