Trawl

MCP Server (Model Context Protocol)

Trawl exposes a Model Context Protocol (MCP) server so LLM clients — Claude Desktop, Cursor, n8n — can discover and call Trawl tools directly, without writing glue code.

Transport: JSON-RPC 2.0 over HTTP, stateless. Two auth paths: session token (Free) or API key (Growth & Pro). Per-action key scopes apply to API keys.

Endpoint

POST https://api.trawl.me/api/mcp
  • Content-Type: application/json
  • Auth: Authorization: Bearer <token> — either a 7-day session JWT (Free) or a trawl_* API key (Growth & Pro)
  • Response is plain JSON (Streamable HTTP, stateless JSON mode — no sticky sessions, no SSE stream).

GET and DELETE return 405 Method Not Allowed in V1.

Auth paths

Both plan tiers use the same header: Authorization: Bearer <token>. The token type determines what's available.

Free Growth & Pro
MCP access Yes — session token Yes — session token or API key
Token type 7-day JWT from trawl login 7-day JWT, or trawl_* key (up to 2-year expiry)
Sent as Authorization: Bearer <jwt> Authorization: Bearer <jwt> or Authorization: Bearer trawl_<key>
Renewal trawl login every 7 days trawl login every 7 days, or set-and-forget with an API key

Free users: MCP is included on the Free plan — no credit card, no API key needed. Use your session JWT (see below).

Growth & Pro users: API keys are the convenient path for stable, unattended agent connections — long-lived, scoped per action, no re-auth. If you have a Growth or Pro plan, create a key under Developers → API Keys and skip to the API key examples. A session JWT works too, on either plan.

Free plan: connect with a session token

Step 1 — get your session token

cURL
npm install -g @trawlme/cli
trawl login
trawl token

Enter your email and password when trawl login prompts. trawl token then prints your stored session JWT straight to stdout — copy it into your MCP client config, or capture it inline:

cURL
export TRAWL_JWT=$(trawl token)

The token is a 7-day JWT. Once it expires, trawl token reports it as expired — run trawl login again to refresh it, then re-run trawl token.

OS fallback: the JWT also lives in the CLI's local config file, in case you need it without the CLI on hand:

  • macOS: ~/Library/Preferences/trawl-cli-nodejs/config.json
  • Linux: ~/.config/trawl-cli-nodejs/config.json (or $XDG_CONFIG_HOME/trawl-cli-nodejs/config.json)
  • Windows: %APPDATA%\trawl-cli-nodejs\Config\config.json

e.g. jq -r .token ~/Library/Preferences/trawl-cli-nodejs/config.json (macOS shown — swap the path for your OS).

Step 2 — configure your MCP client

Pass the token as a Bearer Authorization header. All plans use the same header format — only the token value differs.

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

JSON
{
  "mcpServers": {
    "trawl": {
      "transport": {
        "type": "http",
        "url": "https://api.trawl.me/api/mcp",
        "headers": {
          "Authorization": "Bearer your_jwt_here"
        }
      }
    }
  }
}

Cursor (Settings → Model Context Protocol):

JSON
{
  "mcpServers": {
    "trawl": {
      "url": "https://api.trawl.me/api/mcp",
      "headers": {
        "Authorization": "Bearer your_jwt_here"
      }
    }
  }
}

curl smoke-test (Free path):

cURL
# Verify auth — whoami
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer your_jwt_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"trawl_whoami","arguments":{}}}'

Restart Claude Desktop (or reload Cursor) after editing the config — the trawl server appears in the tool picker.

Re-auth: the session JWT expires after 7 days. Update the Bearer … value in your config after running trawl login again.

Claude Desktop configuration (API key — Growth & Pro)

Add the following block to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

JSON
{
  "mcpServers": {
    "trawl": {
      "transport": {
        "type": "http",
        "url": "https://api.trawl.me/api/mcp",
        "headers": {
          "Authorization": "Bearer trawl_your_key_here"
        }
      }
    }
  }
}

Replace trawl_your_key_here with a key created under Developers → API Keys. Restart Claude Desktop — the trawl server appears in the tool picker.

Cursor configuration (API key — Growth & Pro)

In Cursor settings → Model Context Protocol, add:

JSON
{
  "mcpServers": {
    "trawl": {
      "url": "https://api.trawl.me/api/mcp",
      "headers": {
        "Authorization": "Bearer trawl_your_key_here"
      }
    }
  }
}

n8n / raw HTTP (API key)

Any client that speaks JSON-RPC 2.0 over HTTP works. Here's a bare curl smoke-test:

cURL
# 1. initialize — the client handshake
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'

# 2. tools/list — discover the server's tool surface
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'

# 3. tools/call — invoke whoami to verify auth + scope
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"trawl_whoami","arguments":{}}}'

Tools

Nine tools total, no aliases. Every tool except trawl_whoami and trawl_health_ping requires the scope listed below — a key without it gets -32001 forbidden.

Foundation

Tool Scope Purpose
trawl_whoami (none) Returns userId, email, organizationId, organizationName, scrapIdsAllowlist, mode — used by clients to verify auth + scope post-connect.
trawl_health_ping (none) Lightweight liveness probe. Returns ok, version, uptime (seconds).

Scraps

The four scrap-id-scoped tools honour scrapIdsAllowlist — a scoped API key sees only its allow-listed scraps. Out-of-scope ids return -32001 forbidden; absent ids return -32004 not_found. trawl_create_scrap creates a brand-new scrap rather than referencing an existing one, so the allowlist does not apply to it.

Tool Scope Purpose
trawl_list_scraps trawl:scraps:read List scraps this key can access. Returns title, status (boolean | null from the latest history row — true success, false failure, null never run), and pagination metadata.
trawl_get_scrap trawl:scraps:read Fetch a single scrap by id. Public fields only — encrypted account blobs never leave the server. See Scrap Payload.
trawl_trigger_scrap_run trawl:scraps:run Trigger a scrap run via the same service path as the API worker route. Blocks synchronously until the run finishes (typically 30-250s) — see the client-timeout callout below. Honours existing cron / rate-limit / cost / auto-fix guards. Reports the honest outcome — see below.
trawl_scrap_latest_result trawl:history:read Return the most recent run result for a scrap without re-triggering it. Reads the newest history entry; result/runId are null when the scrap has no history yet.
trawl_create_scrap trawl:scraps:create Generate extraction code via AI from a URL + goal, create the scrap, and trigger its first run (autofix on failure by default). Blocks synchronously (typically 30-250s) — see the client-timeout callout below. Metered — consumes the AI wizard quota. The MCP counterpart of POST /api/ai/wizard.

trawl_list_scraps example

cURL
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc":"2.0",
    "id":"list-1",
    "method":"tools/call",
    "params":{
      "name":"trawl_list_scraps",
      "arguments":{ "limit": 20, "page": 1, "query": "weather" }
    }
  }'

Response payload (inside the single text content block):

JSON
{
  "scraps": [
    {
      "id": "65a1…",
      "title": "Weather — Paris",
      "description": "",
      "url": "https://example.test/weather",
      "status": true,
      "lastRunAt": "2026-04-18T10:12:33.000Z",
      "containerId": "65a0…",
      "cron": "0 */6 * * *",
      "cronTimezone": "UTC",
      "autoFix": false,
      "createdAt": "…",
      "updatedAt": "…"
    }
  ],
  "total": 1,
  "page": 1,
  "perPage": 20,
  "hasMore": false
}

Arguments:

Field Type Default Notes
limit integer 1–100 20 Page size.
cursor string — Opaque keyset cursor from a previous response's nextCursor. Preferred over page — this is the default path whenever page is omitted.
page integer ≥1 — [DEPRECATED] 1-based page index (legacy offset pagination). Passing it opts into the legacy total/page/perPage/hasMore response shape. Will be removed in a future release — use cursor instead.
query string 1–200 — Case-insensitive title/request substring filter.
response_format "json" | "markdown" json "json" = structuredContent + text JSON. "markdown" = human-readable text block.

trawl_get_scrap example

cURL
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc":"2.0",
    "id":"get-1",
    "method":"tools/call",
    "params":{
      "name":"trawl_get_scrap",
      "arguments":{ "scrapId": "65a1f…" }
    }
  }'

Returns { scrap: { id, title, description, url, status, … } } on success, -32001 / -32004 via the JSON-RPC error envelope otherwise.

trawl_trigger_scrap_run example

Blocking call: trawl_trigger_scrap_run runs synchronously and does not return until the run finishes — typically 30-250s. Set your MCP client's tool-call timeout accordingly (many clients default to 30s and will abort mid-run). This is the opposite of the CLI's trawl trigger, which is async by default (returns immediately unless you pass --wait/--watch) — see CLI.

cURL
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc":"2.0",
    "id":"run-1",
    "method":"tools/call",
    "params":{
      "name":"trawl_trigger_scrap_run",
      "arguments":{
        "scrapId": "65a1f…",
        "params": { "city": "Paris" },
        "reason": "manual check after upstream UI change"
      }
    }
  }'

The response reports the honest outcome read back from the run's persisted History row, not a bare success/fail — status is one of:

status Meaning
completed Real success — result holds the scraped data.
blocked The target site blocked the request (DataDome, Cloudflare, etc.).
empty Structurally-valid run that returned no data.
failed The run errored — no runId read-back, error holds the message.

Successful response payload:

JSON
{ "scrapId": "65a1f…", "runId": "65f8…", "status": "completed", "statusDetail": "success", "blocked": false, "length": 12, "result": [ /* scrap data */ ] }

Failure response payload (scrap run itself errored — distinct from a JSON-RPC protocol error):

JSON
{ "scrapId": "65a1f…", "status": "failed", "error": "Upstream timeout" }

runId is the History id of this run — pass it to trawl_get_run for full diagnostics (error, failed selector, proxy tier, auto-fix outcome).

Arguments:

Field Type Required Notes
scrapId string (ObjectId) yes Scrap to run.
params object no Param overrides validated against the scrap's declared param schema.
reason string 1–500 no Free-form audit trail.

trawl_scrap_latest_result example

Returns the most recent run result for a scrap without re-triggering it — reads the newest history entry (newest-first, limit 1).

curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc":"2.0",
    "id":"latest-1",
    "method":"tools/call",
    "params":{
      "name":"trawl_scrap_latest_result",
      "arguments":{ "scrapId": "65a1f…" }
    }
  }'

result and runId are null when the scrap has no history yet. Large payloads (>100KB) are replaced with a REST pointer ({ truncated: true, url }) to GET /api/historys/:id.

Arguments:

Field Type Required Notes
scrapId string (24-char ObjectId) yes Scrap whose latest result to fetch.
response_format "json" | "markdown" no Output format, default json.

trawl_create_scrap example

Generates extraction code via AI from a URL + a goal, creates the scrap, and triggers its first run (autofix on failure by default) — the MCP counterpart of POST /api/ai/wizard, same underlying AiWizardService.runWizard engine.

Blocking call: trawl_create_scrap runs the whole generate → create → run cycle synchronously and does not return until it's done — typically 30-250s. Set your MCP client's tool-call timeout accordingly (many clients default to 30s and will abort mid-run).

curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc":"2.0",
    "id":"create-1",
    "method":"tools/call",
    "params":{
      "name":"trawl_create_scrap",
      "arguments":{ "url": "https://example.test/weather", "goal": "Extract today'\''s temperature and conditions for Paris", "reason": "onboarding demo scrap" }
    }
  }'

success reflects the first run's outcome, not whether the HTTP call succeeded — a false value does not mean scrap creation failed; auto-fix (on by default) retries in the background. historyId is the History id of that first run — pass scrap.id to trawl_scrap_latest_result, or this historyId to trawl_get_run, for diagnostics.

Metered: each call consumes the AI wizard quota (billing). A quota-exhausted org gets -32002 payment_required before any generation runs — see Error shape below.

Arguments:

Field Type Required Notes
url string (URL) yes Target URL to scrape.
goal string 1–500 yes Description of what to extract — drives the AI code generation.
autoFix boolean no Override auto-fix on the created scrap. Default true (self-heals the first run on failure).
reason string 1–500 no Free-form audit trail.

History

Read-only. Large responses cap at 100KB — when a payload would exceed the budget the handler returns { truncated: true, url, reason: 'payload_too_large' } pointing to the REST equivalent.

Tool Scope Input Output
trawl_get_scrap_history trawl:history:read { scrapId: string, from?: ISO8601, to?: ISO8601, limit?: 1..100, cursor?: string } { scrapId, runs: [{ id, scrapId, status, statusDetail, startedAt, endedAt, durationMs, length, triggeredBy, proxyTier, blocked, errorMessage, failedSelector, emptyContext, fixVersionId }], nextCursor }
trawl_get_run trawl:history:read { historyId: string } { run: { …same fields as above… }, autofix: { outcome, classification, reason, fixDiff, dryRunResults, knowledgeUsed, versionId } | null }

trawl_get_scrap_history returns run diagnostics alongside the status and timing — errorMessage, failedSelector, emptyContext (page URL + anchor count), blocked, and proxyTier (abstract Tier 0–4). Cost and proxy nature/vendor are not exposed to non-admin callers by design.

trawl_get_run returns the same diagnostics for a single run plus the full autofix block — the fix decision, unified diff, dry-run results, and knowledge fingerprints consulted by the engine. Use it when you need to understand why a specific run failed or inspect the most recent autofix attempt.

cURL
# Diagnose a single run and its autofix
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"trawl_get_run","arguments":{"historyId":"<objectId>"}}}'

Example trawl_get_run response (inside the single text content block):

JSON
{
  "run": {
    "id": "65f8…",
    "scrapId": "65a1…",
    "status": false,
    "statusDetail": "empty",
    "startedAt": "2026-05-27T08:00:00.000Z",
    "endedAt": "2026-05-27T08:00:12.345Z",
    "durationMs": 12345,
    "length": 0,
    "triggeredBy": "cron",
    "proxyTier": "tier2",
    "blocked": false,
    "errorMessage": null,
    "failedSelector": null,
    "emptyContext": {
      "page": { "url": "about:blank", "totalAnchors": 0 }
    },
    "fixVersionId": null
  },
  "autofix": {
    "outcome": "applied",
    "classification": "selector_fix",
    "reason": null,
    "fixDiff": "@@ -3 +3 @@\n-await page.$$eval('.old-selector', …)\n+await page.$$eval('.new-selector', …)",
    "dryRunResults": [{ "status": "success" }, { "status": "success" }],
    "knowledgeUsed": [{ "fingerprint": "fp-abc123", "confidence": 0.92 }],
    "versionId": "65f9…"
  }
}

trawl_get_run autofix is null when no auto-fix engine ran on that history record.

cURL
# Run history for a scrap over the last 24h
curl -sS https://api.trawl.me/api/mcp \
  -H 'Authorization: Bearer trawl_your_key_here' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"trawl_get_scrap_history","arguments":{"scrapId":"<objectId>","from":"2026-04-17T00:00:00Z","limit":20}}}'

trawl_get_scrap_history honours the API key's scrapIds allowlist in addition to the CASL read Scrap rule. Out-of-scope access returns JSON-RPC -32001 forbidden wrapped in an isError: true tool result.

Auth + scope

The MCP endpoint accepts two credential types, both via Authorization: Bearer:

  • Session JWT (all plans, incl. Free): Authorization: Bearer <jwt> — full org scope, 7-day expiry.
  • API key (Growth & Pro): Authorization: Bearer trawl_<key> — action-scoped, scrap-scoped, up to 2-year expiry. See API Keys for creation and scope reference.

A key restricted to trawl:scraps:read cannot call trawl_trigger_scrap_run. Session tokens carry the caller's full CASL abilities (same as the web interface).

Error shape

All errors conform to JSON-RPC 2.0:

JSON
{ "jsonrpc": "2.0", "id": <id>, "error": { "code": <int>, "message": <string>, "data": <optional> } }

Implementation-defined codes:

Code Meaning
-32602 Invalid params (Zod validation failure)
-32000 Unauthorized
-32001 Forbidden
-32002 Payment required (billing/quota denial — HTTP 402; actionable detail in data, e.g. METER_EXHAUSTED, PAYMENT_PAST_DUE)
-32003 Unavailable (HTTP 503)
-32004 Not found
-32005 Rate limited
-32006 Method not allowed
-32603 Internal error (fall-through)

Rate limiting

Two limiters guard /api/mcp, one running before auth and one after:

Limiter Scope Limit
Pre-auth Per IP — runs before the key/token is checked 60 requests / minute
Per-caller Per identity (user → API key → IP, whichever resolves first) — runs after auth succeeds 120 requests / minute

The pre-auth limiter bounds a failed-auth flood (garbage Bearer tokens, guessed API keys) that never resolves to an identity and would otherwise never touch the per-caller bucket. The per-caller limiter absorbs the JSON-RPC handshake burst (initialize, notifications/initialized, tools/list, each tool call is a separate POST in stateless mode) — it replaces the legacy 1-request/5-second per-key limiter used on classic single-shot REST routes, which does not apply to /api/mcp. Exceeding either returns JSON-RPC -32005 rate_limited.

See also: API Keys · Scrap Payload · Webhooks

Next step → Using Trawl with Claude

Skills
Using Trawl with Claude