Command Palette

Search for a command to run...

UnylyUnyly
Browse all

Crawl4ai2vectordb

FreeNot checked

MCP server for crawling web documentation and storing it in a Supabase vector database.

utaschulz1by utaschulz1
GitHubEmbed

About

MCP server for crawling web documentation and storing it in a Supabase vector database.

README

MCP server for crawling web documentation and storing it in a Supabase vector database. Credits: craw4ai and Cole Medin's mcp server.

Responsibility: ingestion only — crawl, chunk, embed, store. No search, no reranking, no Neo4j.

Tools

Tool Description
crawl_single_page Crawl one URL and store its content
smart_crawl_url Auto-detect URL type (sitemap / txt / webpage) and crawl accordingly
get_available_sources List all sources stored in the database
delete_source Delete a source and all its content from the database

Setup

cp .env.example .env
# fill in OPENAI_API_KEY, MODEL_CHOICE, SUPABASE_URL, SUPABASE_SERVICE_KEY
uv sync

Running

stdio (recommended — Claude Code manages the process):

Add to your MCP config:

{
  "mcpServers": {
    "crawl4ai2vectordb": {
      "type": "stdio",
      "command": "uv",
      "args": ["run", "python", "-u", "src/server.py"],
      "cwd": "/path/to/mcp_crawl4ai2vectordb"
    }
  }
}

or

{
  "mcpServers": {
    "crawl4ai2vectordb": {
      "type": "sse",
      "url": "http://localhost:8051/sse"
    }
  }
}

SSE (manual startup, for multi-client use):

TRANSPORT=sse uv run python -u src/server.py

Optional features

Env var Default Effect
USE_CONTEXTUAL_EMBEDDINGS false Prepend LLM-generated context to each chunk before embedding (improves retrieval accuracy, costs more)
USE_AGENTIC_RAG false Extract code blocks and store them separately in code_examples table

from github.com/utaschulz1/mcp_crawl4ai2vectordb

Installing Crawl4ai2vectordb

This server has no published package — it is built from source. Open the repository and follow its README.

▸ github.com/utaschulz1/mcp_crawl4ai2vectordb

FAQ

Is Crawl4ai2vectordb MCP free?

Yes, Crawl4ai2vectordb MCP is free — one-click install via Unyly at no cost.

Does Crawl4ai2vectordb need an API key?

No, Crawl4ai2vectordb runs without API keys or environment variables.

Is Crawl4ai2vectordb hosted or self-hosted?

A hosted option is available: Unyly runs the server in the cloud, no local setup required.

How do I install Crawl4ai2vectordb in Claude Desktop, Claude Code or Cursor?

Open Crawl4ai2vectordb on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.

Related MCPs

wenb1n-dev/SmartDB_MCP

A universal database MCP server supporting simultaneous connections to multiple databases. It provides tools for database operations, health analysis, SQL optim

wenb1n-devby wenb1n-dev

Postgres Server

This server enables interaction with PostgreSQL databases through the Model Context Protocol, optimized for the AWS Bedrock AgentCore Runtime. It provides tools

madhurprashby madhurprash

Postgres

Query your database in natural language

Anthropicby Anthropic

PostgreSQL

Read-only database access with schema inspection.

modelcontextprotocolby modelcontextprotocol

Redis

Interact with Redis key-value stores.

modelcontextprotocolby modelcontextprotocol

SQLite

Database interaction and business intelligence capabilities.

modelcontextprotocolby modelcontextprotocol

mxcp

Open-source framework for building enterprise-grade MCP servers using just YAML, SQL, and Python, with built-in auth, monitoring, ETL and policy enforcement.

raw-labsby raw-labs

tadas-github/a2asearch-mcp

MCP server to search 4,800+ MCP servers, AI agents, CLI tools and agent skills. Install: npx -y a2asearch-mcp. Ask Claude: "Find MCP servers for database access

tadas-githubby tadas-github

julien040/anyquery

Query more than 40 apps with one binary using SQL. It can also connect to your PostgreSQL, MySQL, or SQLite compatible database. Local-first and private by desi

julien040by julien040

drakonkat/wizzy-mcp-tmdb

A MCP server for The Movie Database API that enables AI assistants to search and retrieve movie, TV show, and person information.

drakonkatby drakonkat

Compare Crawl4ai2vectordb with

Crawl4ai2vectordbvswenb1n-dev/SmartDB_MCPCrawl4ai2vectordbvsPostgres ServerCrawl4ai2vectordbvsPostgresCrawl4ai2vectordbvsPostgreSQLCrawl4ai2vectordbvsRedisCrawl4ai2vectordbvsSQLiteCrawl4ai2vectordbvsmxcpCrawl4ai2vectordbvstadas-github/a2asearch-mcpCrawl4ai2vectordbvsjulien040/anyqueryCrawl4ai2vectordbvsdrakonkat/wizzy-mcp-tmdb

Not sure what to pick?

Find your stack in 60 seconds

Author?

Embed badge for your README

Browse similar

All data MCPs