Scrapers, crawlers and browser agents people built or used with AI to pull data from websites, each linked to its source. 22 matching use cases below, each with a public source.
codyschneider has Claude Code scrape your site into a brand style guide (colors, fonts, logo rules, photo style, tone), study competitors' ads from the Meta Ad Library, then generate statics with GPT Image 2.
See pricing page1 hrIntermediate
isbkch shares a prompt for Cursor, Claude Code or Lovable that fixes AI-built apps shipping as a blank root div, so crawlers get titles, previews and content.
See pricing page30 minIntermediate
ego lite is a browser for agents like Codex and Claude Code that shares your logged-in state; agents work in their own Spaces in parallel while you keep browsing, and it installs as a skill.
See pricing page5 minBeginner
A cold-email operator had his chief-of-staff bot install the Codex CLI on its computer and sign in with ChatGPT, plus an API key for cheaper models. Grok Bot keeps the judgment calls (Calendly booking, CRM follow-ups) and offloads CSV classification, scrapers, spintax and transcript summaries.
$30/mo15 minutesIntermediate
CrawlHive is an open-source memecoin safety tool for Solana: it scores each token on 7 checks (market, holders, bundles, clones, dev, Telegram, X) for a 0 to 100 verdict, reads contract bytecode for 134 dangerous functions, and lets you write your own crawlers in plain language that send signals to Telegram. It also works over MCP and with Grok Bot.
$30/mo1 weekAdvanced
Finding that Hermes Agent is a chat assistant rather than a scraper, a user built her own scraper inside it and asked it to return verified leads.
Free tierDaysIntermediate
The author runs scheduled Cursor cloud agents every day: one scrapes Wispr, Zoom and Gemini meeting transcripts into Obsidian, one triages Gmail, one updates a CRM with new leads, and one builds a morning briefing that Grok Bot prints out.
$30/mo1-2 hoursIntermediate
A retiree has AI write software for his own life. His kids' school app was painfully slow, so he had AI write something that logs in, pulls new updates and feeds them to the AI that manages his calendar and to-do list.
See pricing page10 minBeginner
Built Jevcrawl, an MCP and CLI crawler that finds answer pages more often without calling an LLM while crawling.
$17/mo1-2 hoursIntermediate
open-browser-use lets AI agents control your already logged-in Chrome instead of a fresh browser with no sessions.
See pricing page10 minBeginner
DeepScrape has an LLM generate a CSS selector once, reuses it for free, and calls the model again only when the page changes; it runs as an MCP server for Claude or Cursor and has a Site-to-MCP mode.
See pricing page20 minIntermediate
Open-sourced JobPilot, an MCP server with no LLM of its own that drives your logged-in Chrome for email, LinkedIn Easy Apply and ATS forms and learns from failures.
$17/mo2-4 hoursAdvanced
agent-chrome-relay lets AI agents use your real Chrome, signed in as you, working in background tabs.
See pricing page10 minIntermediate
G Maps Crawl is a remote MCP server that connects Claude Code, Cursor, VS Code or Codex to Google Maps business search and structured listing data.
See pricing page5 minBeginner
Vibe-coded a research agent in Claude Code that scrapes the top TikToks in a niche, has AI break down why they work and writes a brief a team can film.
$17/mohalf a dayIntermediate
Built Browser Harness so an LLM can complete arbitrary browser tasks with fewer guardrail dead-ends.
Free tier1-2 hoursIntermediate
Jev reads a table of a page's controls and picks the operation and target; the demo logs into Hacker News and runs a search in 8 seconds.
See pricing page20 minIntermediate
A geophysicist who lost his job built ai-job-search on Claude Code: it scrapes postings, scores how well each fits him, tailors the CV and cover letter, then preps the interview. He got hired, and the repo picked up 2,232 GitHub stars in a day.
$17/moA dayIntermediate
Built a personal MCP server so Claude can pull Reddit content directly instead of relying on brittle UI scraping.
$17/mo1-2 hoursIntermediate
Agentability is a weekly public experiment where a real AI agent attempts ten everyday web errands and every transcript is published verbatim.
See pricing page10 minBeginner
Markdownee crawls web content into Markdown, text or HTML, built on a TypeScript port of the Trafilatura extractor with Crawlee for crawling.
See pricing page5 minBeginner
Built Nightcrawler, a local AI pentesting agent that runs on a smartphone.
See pricing page1 dayAdvanced