# Give Claude or Cursor the web: MeshArc MCP in 60 seconds

I kept asking my coding assistant about libraries released last month and competitor pricing pages as they stand today, and kept getting confident guesses from memory. So I gave MeshArc a web scraping MCP server: it hands the assistant tools that fetch a page as clean Markdown, list every URL a site declares, crawl a section, and report what changed since the last visit.

This guide connects MeshArc's hosted MCP server to Claude, Claude Code and Cursor. The hosted server has no package to install and no key to copy. Then it shows one real run, with the numbers as they came back: two tool calls, 2 credits.

## What you need

- A MeshArc account. The free plan gives 1,000 credits once, with no card.
- Claude (web, desktop or mobile), Claude Code, Cursor, or any client that accepts a remote MCP URL.
- One URL: `https://mcp.mesharc.dev/mcp`

That is the whole list for the hosted server. If you would rather run the server on your own machine with your own API key, skip to "Run it locally instead".

## Step 1: add the server

### Claude (web, desktop and mobile)

1. Open **Settings → Connectors** and choose **Add custom connector**.
2. Paste `https://mcp.mesharc.dev/mcp` and give it a name.
3. Sign in to mesharc.dev when the browser asks, and approve.

### Claude Code

One line in a terminal:

```bash
claude mcp add --transport http mesharc https://mcp.mesharc.dev/mcp
```

It opens a browser once to sign in and approve, then works.

### Cursor

Use the **Add to Cursor** button on the [MCP page](https://mesharc.dev/mcp?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor), or add the server to `~/.cursor/mcp.json` yourself (`.cursor/mcp.json` inside a repository keeps it to that project):

```json
{
  "mcpServers": {
    "mesharc": {
      "url": "https://mcp.mesharc.dev/mcp"
    }
  }
}
```

Cursor asks you to confirm, then you sign in to mesharc.dev once.

### VS Code and everything else

The MCP page also has an **Add to VS Code** button, which registers the server as `{"type": "http", "url": "https://mcp.mesharc.dev/mcp"}`. ChatGPT connectors and any other client that takes a remote MCP URL use the same address.

## Step 2: approve, and tick the right box

The approval screen matters more than it looks. Approving does not hand the assistant your API key. MeshArc issues the connected app a key of its own, which you can see and revoke under **Settings → API keys**. The assistant never holds a credential.

A connection is **read only unless you allow changes**, and read only means less than most people expect. A read-only connection can look at everything already stored in your workspace (projects, runs, pages, change records), but it cannot reach a website. Anything that fetches counts as a write, because it leaves the workspace and usually spends credits. That includes reading a single page and mapping a sitemap.

So if the goal is to give Claude or Cursor the web, tick **Also allow changes** when you approve. If you only want the assistant to query crawls you have already run, leave it off. A tool called without the right access answers `{"code": "read_only"}` and says what to do about it; nothing fails silently.

## Step 3: ask for a page

Now ask in plain English. Two prompts are enough to see the two tools you will use most.

**"How many pages does the FastAPI docs site declare, and which are in the security tutorial?"** The assistant calls `map_site`, which reads a site's sitemaps without fetching any page.

**"Read the first one and summarise it."** The assistant calls `scrape_urls`, which returns the page as Markdown.

### One real run, with numbers

We ran exactly that through the hosted server on 12 November 2026. Results as they came back:

| Call | What came back | Credits |
| --- | --- | --- |
| `map_site` on `fastapi.tiangolo.com`, filtered to `/tutorial/security/` | 153 URLs declared in 1 sitemap file; 5 under `/tutorial/security/` | 1 |
| `scrape_urls` on `/tutorial/security/first-steps/` | 2,291 words; 127,298 bytes of HTML reduced to 10,400 bytes of Markdown; plain fetch, verdict `ok` | 1 |
| **Total** | The size of the site, the section, and one page read | **2** |

Two details in that table are worth a second look. The map cost 1 credit because it is charged per sitemap file, not per URL, so learning the size of a 153-page site costs the same as learning the size of a 15,000-page one that fits in one file. And the Markdown is about 8% of the HTML by size. That matters inside a chat: the assistant's context holds the content of the page, not its navigation, sponsor banners and script tags.

The same two calls from a script, with the Python SDK:

```bash
pip install mesharc
```

```python
from mesharc import MeshArc

arc = MeshArc()  # reads MESHARC_API_KEY

details = arc.map_details("https://fastapi.tiangolo.com/", search="/tutorial/security/")
print(details["totals"], details["creditsUsed"])

page = arc.scrape("https://fastapi.tiangolo.com/tutorial/security/first-steps/")
print(page["verdict"], page["credits"])
print(page["markdown"][:500])
```

A tool call through MCP is an ordinary authenticated request, so the credits, limits and rate are identical either way.

## Beyond one page: watch a site

Reading a page is where most web MCP servers stop. The larger group of MeshArc's tools is for a site you come back to:

- `crawl_site` crawls a site once, to a limit and depth, with include and exclude globs.
- `create_project` turns a seed URL into a watched project with a schedule: manual, hourly, daily or weekly.
- `start_run` crawls it now, or only the pages you list.
- `get_changes` lists the pages added, modified and removed since the run before.
- `list_pages` searches the stored pages by word, quoted phrase or CSS selector.

The MCP page shows a conversation of this kind as it ran. Asked to "set up a project that only scrapes the blog, weekly", the assistant first called `get_project` with no project, which returns every setting it may set. It then mapped the site, found the blog under `/blogs/` and not `/blog/`, and created the project with `include_paths` set to `["/blogs/*"]`. The first run read 30 blog pages and nothing else.

Setting names are exact. A guessed key such as `includePaths` is refused with its name, so the assistant corrects itself and does not go on believing a project is configured when it is not. The full list is on the [configuration page](https://mesharc.dev/docs/configuration?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor).

Reading what is stored is free. `list_projects`, `list_runs`, `list_pages`, `get_changes` and `get_job` cost 0 credits, so "what changed on the docs since Monday?" costs nothing to ask as often as you like. If the pages are feeding a retrieval index, [how to keep a RAG knowledge base fresh](https://mesharc.dev/blog/keep-rag-knowledge-base-fresh?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor) covers the other half, and the [RAG pipeline use case](https://mesharc.dev/use-cases/rag-pipeline?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor) shows the whole route from crawl to vector store.

## What it costs

There is no charge for the MCP server itself. You pay for the work, at the same rates as the API (pricing page, checked 11 November 2026):

| Work | Credits |
| --- | --- |
| A page read by plain fetch | 1 |
| A page that needs a real browser | 4 |
| A sitemap file read by `map_site` | 1 |
| A page the site refused, a 404 or a 410 | 0 |
| Listing projects, runs, pages and changes | 0 |

On the Growth plan ($99 a month, 500,000 credits) a plain page works out at $0.20 per 1,000. The free plan's 1,000 credits cover a lot of chat-sized questions: the run above used 2. Every response states what it cost. Full rates are on [pricing](https://mesharc.dev/pricing?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor).

## Run it locally instead

Some teams do not connect hosted apps, or want the server to hold a narrowly scoped key. The server ships in the Python package (Python 3.10 or newer) and speaks stdio:

```bash
pip install "mesharc[mcp]"
claude mcp add mesharc -e MESHARC_API_KEY=mesharc_... -- mesharc-mcp
```

For Claude Desktop or Cursor, the same thing as a config entry:

```json
{
  "mcpServers": {
    "mesharc": {
      "command": "mesharc-mcp",
      "env": { "MESHARC_API_KEY": "mesharc_..." }
    }
  }
}
```

Make the key under **Settings → API keys**. A `read` key lets the assistant look at what is stored; `write` lets it fetch and run projects. A key can also be limited to named projects, and given an expiry. One difference from hosted: deleting a project needs an admin key, which a connected app never holds, so `delete_project` only works locally.

## What it will not do

Better to know before the assistant finds out mid-conversation.

| Limit | What happens |
| --- | --- |
| Pages behind a sign-in | The crawler reads what a visitor can see. A login wall comes back as blocked, at 0 credits |
| Checking a page every minute | Schedules are hourly at the finest. `start_run` with the pages listed is the on-demand check |
| A 5,000-page crawl inside one chat turn | Hosted, a tool that would hold the connection open hands back a job after 45 seconds; `get_job` picks it up. Start a big site as a project and read it back with `list_pages` |
| A very long page in full | One page asked for on its own is capped at 12,000 characters in the tool's answer. The API returns the whole page |
| A read-only connection that fetches | Refused with `read_only` until you reconnect with changes allowed |
| More than the plan allows | The same credits and limits as a script, and the same 402 when they run out |

If all you need is one page fetched on demand and never revisited, a single-purpose fetch tool is less to set up, and your assistant may already have one built in. This server earns its place when the assistant should remember a site and tell you what moved.

## The short version

- Add `https://mcp.mesharc.dev/mcp` as a connector in Claude, with `claude mcp add --transport http` in Claude Code, or in `mcp.json` in Cursor.
- Sign in and approve. Hosted server: no key to copy. Tick **Also allow changes** if the assistant should fetch pages.
- `map_site` sizes a site for 1 credit per sitemap file; `scrape_urls` reads a page for 1 credit by plain fetch or 4 with a browser.
- Projects, runs and `get_changes` turn "read this page" into "tell me what changed".
- Refused pages, 404s and reads of stored data cost nothing.

Try a URL in the [playground](https://mesharc.dev/playground?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor) to see what the assistant will get back from it, or [start free](https://mesharc.dev/signup?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor) with 1,000 credits and no card, then paste the server URL into your assistant.

---

*Originally published on* [*mesharc.dev*](https://mesharc.dev/blog/web-scraping-mcp-claude-cursor?utm_source=hashnode&utm_medium=referral&utm_campaign=2026w47-ladder&utm_content=web-scraping-mcp-claude-cursor)*.*
