Agent Reach: The Complete Guide to Giving Your AI Agent Eyes on the Entire Internet
Your AI agent can write code, manage projects, and edit documents — but ask it to check what people are saying on Twitter about your product, summarize a YouTube tutorial, or scan Reddit for bug reports? It's blind. Agent Reach is the open-source CLI scaffolding tool that solves this problem with a single install command: pip install agent-reach. With 6,400+ GitHub stars and support for 13 platforms including Twitter, Reddit, YouTube, Bilibili, XiaoHongShu, LinkedIn, and more — all with zero API fees — Agent Reach is the fastest way to give any AI coding agent full internet access.
Why Agent Reach Exists
AI agents are incredibly capable within their sandbox — they can modify files, run commands, analyze code. But the moment you need them to interact with the internet, everything breaks:
- 📺 "Summarize this YouTube tutorial" → Can't extract subtitles
- 🐦 "Search Twitter for product feedback" → Twitter API costs $100+/month
- 📖 "Check Reddit for similar bugs" → 403 Forbidden — server IPs blocked
- 📕 "Read this XiaoHongShu post" → Login required, can't access
- 📺 "Summarize this Bilibili video" → Blocked outside China
- 🔍 "Search for the latest LLM frameworks" → No quality free search
- 🌐 "Read this webpage" → Returns raw HTML soup, not readable text
- 📦 "What does this GitHub repo do?" → Works but auth is painful to configure
Each platform has its own barriers — paid APIs, IP bans, login walls, dirty data. Agent Reach consolidates all of these into a single install:
pip install agent-reach
agent-reach install
After that, your agent can read tweets, search Reddit, extract YouTube subtitles, browse XiaoHongShu, and more — automatically.
Supported Platforms (13 Channels)
| Platform | Tool Used | Auth Required | What It Does |
|---|---|---|---|
| Web (any URL) | Jina Reader | ❌ | Converts any webpage to clean, LLM-readable markdown |
| Twitter/X | xreach CLI | 🍪 Cookie | Search tweets, read threads, analyze accounts |
| YouTube | yt-dlp | ❌ | Extract video metadata, subtitles, multi-language support |
| JSON API + Exa | ❌ (proxy on servers) | Search posts, read threads, community analysis | |
| GitHub | gh CLI | ❌ | View repos, search code, read issues and discussions |
| Bilibili | yt-dlp | ❌ | Video metadata and subtitle extraction from China's YouTube |
| XiaoHongShu | mcporter MCP | 🍪 Cookie (Docker) | Read notes, search content on China's Instagram |
| Douyin/TikTok | mcporter MCP | ❌ | Parse video info, get watermark-free downloads |
| linkedin-mcp | 🍪 Cookie | Profile and company page scraping | |
| BossZhipin | mcp-bosszp | 🍪 Cookie | Job listing search (Chinese job platform) |
| WeChat Articles | camoufox + miku | ❌ | Search and read WeChat public account articles |
| RSS | feedparser | ❌ | Subscribe to and parse RSS feeds |
| Exa Search | mcporter MCP | ❌ | AI-powered semantic search across the internet |
Quick Start
Installation (One Command)
Copy-paste this to your AI agent (Claude Code, Cursor, OpenClaw, etc.):
Help me install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
The agent handles everything: installs the CLI, system dependencies (Node.js, gh CLI, mcporter, xreach, etc.), configures the search engine (Exa, free), and registers a SKILL.md so the agent knows which tool to call for what.
What the Installer Does
- Installs CLI —
pip install agent-reach - Installs system dependencies — Automatically detects and installs Node.js, gh CLI, mcporter, xreach, etc.
- Configures search engine — Exa via MCP (free, no API key)
- Detects environment — Local laptop vs. server, provides appropriate setup
- Registers SKILL.md — Agent learns when to call which tool
Health Check
agent-reach doctor
Shows the status of every channel — which ones are working, which need configuration.
Design Philosophy: Scaffolding, Not Framework
Agent Reach is a scaffolding tool — it sets up the tools your agent needs, then gets out of the way. After installation, the agent calls the upstream tools directly (xreach, yt-dlp, gh CLI, etc.). There's no wrapper layer.
This matters because:
- No lock-in — don't like a tool? Replace the channel file
- No overhead — agents call tools directly, not through Agent Reach
- No dependency — uninstall Agent Reach and the tools still work
- No maintenance — when platforms change, the upstream tools update, not your code
Pluggable Architecture
Every platform is a separate channel file:
channels/
├── web.py → Jina Reader ← swap for Firecrawl, Crawl4AI…
├── twitter.py → xreach ← swap for Nitter, official API…
├── youtube.py → yt-dlp ← swap for YouTube API, Whisper…
├── github.py → gh CLI ← swap for REST API, PyGithub…
├── bilibili.py → yt-dlp ← swap for bilibili-api…
├── reddit.py → JSON API + Exa ← swap for PRAW, Pushshift…
├── xiaohongshu.py→ mcporter MCP ← swap for other XHS tools…
├── douyin.py → mcporter MCP ← swap for other Douyin tools…
├── linkedin.py → linkedin-mcp ← swap for LinkedIn API…
├── rss.py → feedparser ← swap for atoma…
├── exa_search.py → mcporter MCP ← swap for Tavily, SerpAPI…
└── __init__.py → channel registry (doctor detection)
Each channel file only does one thing: check if its upstream tool is available via a check() method. The actual scraping/reading is done by the agent calling the upstream tool directly.
Usage Examples (After Installation)
No commands to memorize. Just tell your agent:
| What You Say | What Happens |
|---|---|
| "Read this link" | curl https://r.jina.ai/URL → clean markdown |
| "What is this GitHub repo?" | gh repo view owner/repo |
| "Summarize this video" | yt-dlp --dump-json URL → subtitles |
| "Check this tweet" | xreach tweet URL --json |
| "Subscribe to this RSS" | feedparser parses the feed |
| "Search GitHub for LLM frameworks" | gh search repos "LLM framework" |
| "Search Twitter for product reviews" | xreach search "query" --json |
The agent reads the installed SKILL.md and knows which tool to call.
Security
Safe Mode
agent-reach install --safe # Won't auto-install system packages
agent-reach install --dry-run # Preview only, no changes
Cookie Security
⚠️ Use a burner account for platforms requiring cookies (Twitter, XiaoHongShu). Reasons:
- Ban risk — platforms may detect non-browser API calls
- Credential scope — cookies equal full login access; limit damage with throwaway accounts
Cookies are stored locally at ~/.agent-reach/config.yaml. They never leave your machine.
Uninstall
agent-reach uninstall # Full cleanup
agent-reach uninstall --dry-run # Preview
agent-reach uninstall --keep-config # Keep tokens, remove skills
pip uninstall agent-reach # Remove Python package
Agent Reach vs Alternatives
Category: This repo is a CLI scaffolding tool for AI agent internet/web access. Alternatives must also be tools that give AI agents the ability to read, scrape, or search the web.
| Feature | Agent Reach | Firecrawl | Crawl4AI | Jina Reader |
|---|---|---|---|---|
| Focus | Multi-platform agent setup | Web data API for AI | LLM-friendly web crawler | URL to markdown |
| Platforms Covered | 13 (Twitter, Reddit, YouTube, etc.) | Web pages only | Web pages only | Web pages only |
| Social Media Access | ✅ Twitter, Reddit, XiaoHongShu, LinkedIn | ❌ | ❌ | ❌ |
| Video Support | ✅ YouTube, Bilibili, Douyin | ❌ | ❌ | ❌ |
| Chinese Platforms | ✅ Bilibili, XiaoHongShu, Douyin, WeChat, BossZhipin | ❌ | ❌ | ❌ |
| API Cost | ✅ $0 (free) | Free tier + paid | ✅ $0 (self-hosted) | Free tier + paid |
| Output Format | Varies by channel | ✅ Markdown + JSON + structured | ✅ LLM-ready markdown | ✅ Clean markdown |
| Structured Extraction | ❌ (raw data) | ✅ AI-powered schemas | ✅ Heuristic + LLM extraction | ❌ |
| Browser Automation | ❌ | ✅ Browser sandbox | ✅ Full browser control | ❌ |
| Self-Hosted | ✅ | ✅ | ✅ Easy Docker | ✅ |
| MCP Integration | ✅ mcporter | ✅ | ✅ | ❌ |
| Community | 6.4K ⭐ | 88.8K ⭐ | 61.5K ⭐ | 10.1K ⭐ |
| Language | Python | TypeScript | Python | TypeScript |
When to choose Agent Reach: You want one install command that gives your AI agent access to 13+ platforms — social media, video, Chinese platforms, search, RSS — with zero API fees. The only tool that covers Twitter + Reddit + YouTube + Bilibili + XiaoHongShu in one package.
When to choose Firecrawl: You need high-quality, structured web scraping with AI-powered data extraction, browser sandboxes, and enterprise-grade reliability for web pages specifically.
When to choose Crawl4AI: You need a powerful, self-hosted web crawler that converts pages to LLM-ready markdown with advanced features like session management, proxies, and browser automation.
When to choose Jina Reader: You want the simplest possible way to convert any URL to clean markdown — just prefix with https://r.jina.ai/ and you're done. (Note: Agent Reach uses Jina Reader as its web channel.)
FAQ
Does it work with my AI agent?
Yes. Any agent that can run shell commands: Claude Code, Cursor, OpenClaw, Windsurf, Codex, and more.
Is it really free?
100% free. All backends are open-source (xreach CLI, yt-dlp, Jina Reader, Exa, etc.). Only optional cost: ~$1/month for a residential proxy if you need Reddit/Bilibili from a server.
How do I add Twitter?
Install Agent Reach, export your Twitter cookies with the Cookie-Editor Chrome extension, run agent-reach configure twitter-cookies "your_cookies". Done.
What about Reddit blocking?
Local computers work fine. Servers need a residential proxy (~$1/month): agent-reach configure proxy http://user:pass@ip:port.
Can I add new channels?
Yes — each channel is a single Python file. Clone the repo, add your file, or submit a PR.
Conclusion
Agent Reach is the missing layer between AI agents and the internet. While tools like Firecrawl and Crawl4AI excel at web scraping, Agent Reach is the only tool that bundles access to 13 platforms — from Twitter and Reddit to Bilibili and XiaoHongShu — into a single pip install. Its scaffolding philosophy means no lock-in: install it, let your agent learn the tools, and the upstream tools work independently.
With 6,400+ stars and active maintenance, Agent Reach is the fastest way to stop your AI agent from being blind to the internet.
Resources
- GitHub Repository
- English Documentation
- Installation Guide
- Upstream Tools: Jina Reader, xreach, yt-dlp, Exa, mcporter, gh CLI, feedparser