capability
Scraper agents
This page lists every AI agent in the MeshKore directory tagged with the Scraper capability. Agents are sourced from public platforms (GitHub, Hugging Face, npm, PyPI, awesome-list curations, and direct submissions), normalized by the MeshKore worker, and ranked by GitHub stars. Each card links to the agent's profile with details on capabilities, framework, language, freshness, and source attribution.
132 agents in this capability · ranked by popularity
Top 132 Scraper agents
🔥 The Web Data API for AI - Power AI agents with clean web data
Create agents that monitor and act on your behalf. Your agents are standing by!
AIHawk aims to easy job hunt process by automating the job application process. Utilizing artificial…
Python scraper based on AI
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili…
Turn any webpage into structured data using LLMs
A community-driven way to read and chat with AI bots - powered by chatGPT.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP…
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation…
Web crawler and scraper for Rust
Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator
Get clean data from tricky documents, powered by vision-language models ⚡
Open-source MCP server for LinkedIn. Give Claude and any MCP-compatible AI assistant access to profiles…
Linkedin Automation Tool: Describe your product. Define your target market. The AI finds the leads for you.
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request…
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps…
🔥 This repository contains complete application examples, including websites and other projects, developed…
小红书版Openclaw，自媒体创作者的AI工作台，小红书创作AI工具RedClaw，支持小红书图文下载、创作风格学习、智囊团AI群聊、小红书AI创作，小红书内容打包下载等创作全程AI化，AI图文制作，AI文章排版，AI…
AI Scraper is a powerful scraping tool and scrape agent built to automate data extraction with unmatched…
Open-source, production-grade web scraping engine built for LLMs. Scrape and crawl the entire web, clean…
Official implement of paper "AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation"…
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust…
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page…
Resume_Builder_AIHawk is a powerful Python tool that allows you to automatically customize your resume based…
ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.
OpenClaw-inspired autonomous AI agent built entirely in n8n. Adaptive RAG-powered memory, Skills via MCP…
Use LLMs to robustly extract web data
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an…
AI agent that can SEE 👁️, control, navigate, & do stuff for you on your browser.
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A…
Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web…
High-performance web crawler API optimized for LLMs. Turn any search or website into clean Markdown using…
Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
This project provides a powerful web scraping tool that fetches search results and converts them into…
Developed an AI application using LLM to analyze user resumes and provided the summarization, strengths…
⚡ The Complete X/Twitter Automation Toolkit — Scrapers, MCP server for AI agents (Claude/GPT), CLI, browser…
Unofficial Claude API supporting direct HTTP chat creation/deletion/retrieval, messages with multiple file…
AI tool for automating Upwork job applications using AI agents to find and qualify jobs, write personalized…
A powerful MCP server extension providing web search and content extraction capabilities. Integrates…
Parse XML sitemaps and extract URLs. Designed to process millions of URLs while bypassing most modern…
Open‑source alternative to Perplexity Comet, director.ai and firecrawl combined
wxpath - declarative web crawling with XPath; a Web Query Language (WQL)
Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation…
ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on…
linkedin-jobs-RAG
AI web scraper built with Crawl4AI for extracting structured leads data from websites.
Extract data from websites in LLM ready JSON or CSV format. Crawl or Scrape entire website with Website…
MCP server for scraping LinkedIn, Facebook, Instagram profiles and Google search.
RAG Web Browser is an Apify Actor to feed your LLM applications and RAG pipelines with up-to-date text…
⚡ Build structured YouTube datasets at scale — effortlessly fetch transcripts and rich metadata for NLP, ML…
Reddit_Commentator_AIHawk is a Python project showcasing the power of artificial intelligence in social media…
This repo provides guidance on setting up a bedrock agent to webscrape and internet search via action groups
A simple, easy to use framework for adding randomized, anonymous IP addresses and user-agents to web…
Fast, lightweight Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI…
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation…
Retrieval-augmented generation (RAG) for remote & local LLM use
Modern JAV metadata manager — multi-source scraping, Jellyfin integration, and AI-ready API. Built with…
Reddit AI Agent is an intelligent tool that helps you explore Reddit like never before! 🔎 It allows you to…
Create your own Alibaba dataset and interact with it in plain English.
A chatbot demo that scrapes a website and stores the result in a vector db, which can then be queried via…
Build web scraping agents using AI to auto-extract the data from websites, capture screenshot, generate pdf…
ChatGPT selenium scraper written in Python
Leverage modern open-source tools to create better web scraping workflows.
A tool to scrape all files from a GitHub repository and turn it into a JSON or TXT file, Useful for AI and…
How to guides on web-crawling or scraping
This is a template repository for building a web scraper with OpenAI support. The repository provides a basic…
Just mention want you want and it will extract/scrape data from the Web. Useful to create AI web…
[IEEE S&P'26] WebCloak: Characterizing and Mitigating the Threats of LLM-Driven Web Agents as Intelligent…
A :robot: which provides features from Wikipedia like summary, title searches, location API etc.
Parse SaaS pricing page using Open AI - GPT-3.5
Python, Javascript, and Rust libraries for the Spider Cloud API.
AI-powered deep research tool leveraging web scraping for cost-effective, comprehensive analysis. Open-source…
⚡️ Real-time Knowledge Graph for AI Agents. Connect LLMs to verified weather, stock, and currency data via…
A Python project that extracts data from websites with the option to process the data through @openai's…
The topic is about product matching via Machine Learning. This involves using various machine learning…
🧘 Reddit-powered AI optimization tips | Save your time and credits | can cover 380+ services | Real tips from…
AURORA (Artificial Unified Responsive Optimized Reasoning Agent) uses lobes and web research for RAG based…
Webpage to structured data in Rust & LLM
Instant, local access to complete Base44 documentation with AI assistant integration
MCP server for Olostep — the web scraping, crawling, and search infrastructure used by top AI companies…
Efficient RAG knowledge pack creator from online Julia documentation
RAG-based Web Scraping
A collection of intelligent AI agents built using Ollama, LangChain, and local LLMs — including chatbots…
Self-hosted multi-agent AI platform, orchestrate specialized agents across Telegram, WhatsApp, and web with…
Scrapes headlines from CNN and FOX, then has ChatGPT do cross-analysis
Claude Code Skill - 抓取微信公众号文章并转换为 Markdown，自动下载图片 | WeChat Article to Markdown Converter
Grabs all your Perplexity conversations data, spits it out into a nice file folder structure and allows you…
钉钉智能机器人，支持AI问答、知识库检索、JIRA管理和服务器维护、周报日报总结、快捷创建工单等等
Telegram bot which helps in promoting Instagram accounts
Telegram bot utilizing OpenAI's GPT to generate presentations and abstracts in PPTX and DOCX formats.
AI-powered job search plugin for Claude Code — multi-platform scraping, visa checks, salary analysis, ranked…
Retrieval-augmented docs ingestion stack: Firecrawl + Crawl4AI + Qdrant vector search with FastAPI and MCP…
This is the repository for a Streamlit application that helps with job applications. This app integrates…
Scrapes Reddit threads and sends content to chosen AI chatbot for analysis.
🦅 DestinyScout: 一款基于 Agent-Native LLM + Boss直聘的 L3 级自主个性化求职引擎。告别机械搬运，它能深度注入你的私人职场 DNA…
A messenger bot that answers messages by scraping stackoverflow questions and answers
A bot that scrapes open-interest and liquidation heatmaps to alert traders when a "Short Squeeze" or "Long…
A collection of intelligent AI agents built using Ollama, LangChain, and local LLMs, including chatbots…
PromoBot - A web scraper that monitors promotion sites by searching keywords and reporting to a Telegram…
Python implementation of https://github.com/mishushakov/llm-scraper
The Real Time Social Media Content Retrieval System fetches real-time LinkedIn posts based on user queries…
🕷️ Open-source web crawling toolkit — Video, OCR, NLP, Stealth, 10+ parsers
This project is an automatic web scraper that uses the LLM Ollama gpt-oss:20b to parse the body content of a…
A powerful RAG tool that scrapes YouTube channel videos, extracts transcripts, and enables AI-powered chat…
Lightfeed SDK to search and filter web data
A Python tool that automates LinkedIn job search, ranking, and export by combining Bright Data's LinkedIn Job…
An intelligent, universal web scraper powered by Google Gemini AI. Features intent-based data extraction…
Tracking the systems that automate scientific research — from literature scrapers to full paper-writing…
Tired of LLMs citing fake papers? renderscholar is a Google Scholar scraper (inspired by Andrej Karpathy’s…
Claude Code settings.json auto-config tool to quickly switch API_KEY, AUTH_TOKEN, and model configs across…
A simple npm package to perform requests as a user on the OpenAI ChatGPT page.
Create stunning media with our AI-powered app. Generate images and videos from text and images using advanced…
This AI bot goes online, gathers information about AI startups, and posts updates about them on X and Dev.to.
A lightweight web browser for humans (CLI/TUI) and AI agents (MCP)
AI-first web scraping engine with stealth bypass, MCP server, and multimodal output (Markdown, JSON, PDF) for…
Infrastructure layer for AI agent swarms — 88 MCP tools · A2A · OmniMesh VPN · Scrapling scraper · COC sync ·…
Stone Scraper is an AI-powered tool for automated web data extraction. Built with Streamlit, Langchain, and…
LinkedEdge: Unlock Your Interview Success with LinkedEdge, your AI-powered job interview preparation…
This is the repository for a Streamlit application that helps with job applications. This app integrates…
🏠 Автоматический мониторинг недвижимости с ИИ-анализом | Парсинг Avito и ЦИАН | Telegram-бот | Docker | Deno
Integrating OpenAI Agents SDK with Bright Data Web Unlocker, enabling AI agents to access, extract, and…
Smart Job Assistant is a simple and intelligent web app that helps job seekers find the right opportunities…
This is a Python - based application that allows the user to search for information and open URLs.
This python powered AI based RAG Scraper allows you to ask question based on PDF/URL provided to the software…
AI News CLI: Search, scrape, and fact-check global news from your terminal. Features local LLM support…
This MCP server enables LLMs to retrieve and process web scraping requests using ScraperAPI.
📰 Discover and summarize top news stories effortlessly with the News Aggregator AI Agent, saving you time…
A Student hub, real-time plugin based web-application featuring a chat, marketplace, video conferencing, and…