Go web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data).

This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.

MCP server tailored to connecting web crawler data and archives

🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.

An MCP server providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content.

Quickly reads webpages and converts to markdown for fast, token efficient web scraping

Model Context Protocol (MCP) Server for Graphlit Platform