scrape-le-mcp

by nolindnaidoo · Search & Web · MCP server

scrape-le-mcp is an MCP server by nolindnaidoo. Analyse robots.txt content and report whether a path may be crawled. Its npm package was downloaded 227 times last week.

Category: Search & Web
Author: nolindnaidoo
GitHub stars: Not on GitHub
This week: Tracking since 18 Oct 2026
Weekly downloads: 227
Language: JavaScript
License: MIT (open source)
Last update: 2026-10-17 (7 days ago)
Links: Source npm

mcp
modelcontextprotocol
model-context-protocol
mcp-server
ai-agent
claude
cursor
vscode
llm-tools
scrape-le
scrape-le-mcp
nolindnaidoo
+1 more

How to install scrape-le-mcp

scrape-le-mcp is published on npm as scrape-le-mcp, and it is listed in the official MCP Registry. These steps assume it runs as a local stdio server, which is the usual setup.

Check the README. Confirm the start command and note any API keys or environment variables it needs (pass them with -e KEY=value or an env block). You need Node.js for npx.

Add it to Claude Code from a terminal:

claude mcp add scrape-le-mcp -- npx -y scrape-le-mcp

Or add it to Cursor or Claude Desktop in .cursor/mcp.json or claude_desktop_config.json:

{
  "mcpServers": {
    "scrape-le-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "scrape-le-mcp"
      ]
    }
  }
}

Restart the client and check it. Run claude mcp list, or open the client's MCP settings, and confirm scrape-le-mcp and its tools are listed. The install guide covers VS Code and Codex.

Also listed in

Maintain this project? Get it featured ($99/mo, pinned at the top of Search & Web) or see what featuring includes.

Report a problem with this listing

Frequently asked questions

What is scrape-le-mcp?

scrape-le-mcp is an MCP server by nolindnaidoo in the Search & Web category. Analyse robots.txt content and report whether a path may be crawled. Its npm package was downloaded 227 times last week. It is written in JavaScript. It is licensed under MIT. It was last updated 7 days ago.

Is scrape-le-mcp free and open source?

Yes. scrape-le-mcp is published under the MIT license, so you can run, modify and self-host it at no cost. If it connects to a paid service, that service's own pricing and API terms still apply. Check the repository for the exact license text before using it commercially.

What are the alternatives to scrape-le-mcp?

Other Search & Web projects on MCP Harbor include Scrapling, TrendRadar, repomix. Each alternative links to its own page with stars, license and recent activity. The Search & Web category ranks all 746 by stars, and you can re-sort it by growth this week.

Alternatives to scrape-le-mcp

ScraplingTrending★ 89,376

D4Vinci · Python+1,131 stars this week (+1.3%)

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

TrendRadar★ 63,148

sansan0 · Python+171 stars this week (+0.3%)

⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载，你的 AI 舆情监控助手与热点筛选工具！聚合多平台热点 + RSS 订阅，支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机，也支持接入 MCP 架构，赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ，数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。

repomix★ 29,005

yamadashy · TypeScript+131 stars this week (+0.5%)

📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.

open-seoTrending★ 24,721

every-app · TypeScript+829 stars this week (+3.5%)

Open source alternative to Semrush and Ahrefs

apify-mcp-serverTrending★ 12,865

apify · TypeScript+1,179 stars this week (+10.1%)

The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.

Horizon★ 10,982

Thysrael · Python+347 stars this week (+3.3%)

📡 Your own AI-powered news radar. Generates daily briefings in English & Chinese. | 用 AI 构建你专属的新闻雷达

All 746 Search & Web MCP servers →

Data last updated 25 October 2026. Stars and downloads are refreshed every day from GitHub and npm.