Most users won't write raw HTTP — the CLI (trawl login, see @trawlme/cli) and MCP server (Claude Desktop / Cursor, see MCP Server) cover day-to-day integration. API keys are the shared authentication layer underneath both, and the direct path when you need plain REST.

This guide covers: creating a key, what scopes grant, scoping a key to specific scraps, and the rate limits that apply.

Get an API key

Create an API key from the Trawl web app under Developers → API Keys. Click Create, choose a name and an expiry date, then copy the key immediately — it is shown only once.

Over the API, only name is required — omit expiresAt and the key defaults to a 1-year expiry:

curl -X POST https://api.trawl.me/api/developers/keys \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{ "name": "ci-pipeline" }'
# → 200, expiresAt set to 1 year from now

Format: trawl_ followed by 40 characters.

Expiry policy

Every API key carries an expiry date — the web app always asks for one, and the API applies a sensible default when you don't:

• expiresAt is optional on creation. Omit it and the key defaults to 1 year from the creation date.
• When you do provide it, the expiry must be in the future and at most 2 years from the creation date. Anything past that window is rejected with 422.
• This limits the blast radius of a leaked key by forcing rotation on a predictable horizon.
• Legacy keys created before this policy (with no expiry) keep working until you explicitly revoke them; only new creates are validated against the rules above.

Pick the shortest expiry that matches your use case (e.g. 3 months for an experiment, 1 year for a long-running integration). When a key expires you can create a fresh one and revoke the old one from Developers → API Keys.

Authenticate

Pass the key in the Authorization header on every request:

Authorization: Bearer trawl_your_key_here

See the Rate limits section below for the limits applied to API-key traffic.

Available endpoints

Method	Path	Required scope	Description
GET	/api/scraps	trawl:scraps:read	List scraps of the owning org
GET	/api/scraps/:id	trawl:scraps:read	Get a single scrap
POST	/api/scraps/worker/:id	trawl:scraps:run	Trigger a scrap run
GET	/api/historys	trawl:history:read	Read scrap history (filter with ?scrap=)

Query params for GET /api/scraps: search, page, perPage.

Creation, update, and deletion of scraps remain UI-only (JWT-authenticated sessions). Any write mutation (create, update, delete) via an API key will be rejected with 401 Unauthorized.

Action scopes

API keys carry an optional scopes[] array that narrows which actions the key can perform. Scopes compose with the resource-level scrapIds[] scoping documented below — a key needs both the right action scope and the right scrap id to succeed.

Scope	Grants
trawl:scraps:read	List scraps, read one scrap
trawl:scraps:write	Reserved — write mutations stay UI-only today
trawl:scraps:run	Trigger a scrap run (POST /api/scraps/worker/:id)
trawl:history:read	Read scrap history (GET /api/historys)
trawl:tasks:read	Reserved — tasks endpoints are JWT-only today
trawl:webhooks:manage	Reserved — webhooks management is JWT-only today
trawl:api-keys:manage	Reserved — API-key management is JWT-only today

Scope naming follows the OAuth 2.1 pattern {vendor}:{resource}:{action}, so the same shape can be mapped to future OAuth scopes without a breaking change on the API.

Default: all scopes granted

If you omit scopes at creation — or pass an empty array — the key has all actions granted (scoped access): every action is granted. This keeps existing integrations working untouched.

Create a scope-restricted key

curl -X POST https://api.trawl.me/api/developers/keys \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "reader-only",
    "expiresAt": "<optional ISO 8601 timestamp, max 2 years from now; defaults to +1 year>",
    "scopes": ["trawl:scraps:read", "trawl:history:read"]
  }'

Out-of-scope request → 403

curl -X POST https://api.trawl.me/api/scraps/worker/64a1f2e3b5c6d7e8f9a0b1c2 \
  -H "Authorization: Bearer trawl_reader_only_key"
# → 403 Forbidden
# WWW-Authenticate: Bearer error="insufficient_scope", scope="trawl:scraps:run"
# {
#   "type": "error",
#   "message": "Forbidden",
#   "description": "API key missing required scope(s): trawl:scraps:run"
# }

The WWW-Authenticate header follows RFC 6750 so clients can parse the missing scope automatically.

Scoping keys to specific scraps

By default, an API key has organization-wide access — it can read and trigger every scrap owned by the organization it was issued under. You can optionally restrict a key to a subset of scraps by passing a scrapIds array at creation time.

scrapIds value	Access
[] (or omitted)	Organization-wide — every scrap in the org. Default. Fully back-compatible with pre-existing keys.
["<scrapId>", …]	Restricted — only the listed scraps are reachable. Every other scrap returns 403 Forbidden.

Each entry must be a 24-character hex Mongo ObjectId belonging to the caller's organization. Foreign or unknown ids are rejected at creation time.

Create a scoped key

curl -X POST https://api.trawl.me/api/developers/keys \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prices-only",
    "expiresAt": "<optional ISO 8601 timestamp, max 2 years from now; defaults to +1 year>",
    "scrapIds": ["64a1f2e3b5c6d7e8f9a0b1c2", "64a1f2e3b5c6d7e8f9a0b1c3"]
  }'

The response is identical to an org-wide key — plainKey is shown once. The key now only accepts requests targeting the two listed scrap ids.

In-scope request → 200

curl https://api.trawl.me/api/scraps/64a1f2e3b5c6d7e8f9a0b1c2 \
  -H "Authorization: Bearer trawl_your_scoped_key"
# → 200 OK, scrap payload

Out-of-scope request → 403

curl https://api.trawl.me/api/scraps/64a1f2e3b5c6d7e8f9a0b1c9 \
  -H "Authorization: Bearer trawl_your_scoped_key"
# → 403 Forbidden
# {
#   "type": "error",
#   "message": "Forbidden",
#   "description": "API key not scoped to this scrap"
# }

The 403 applies to every scrap route that targets a single scrap by id — GET /api/scraps/:scrapId and POST /api/scraps/worker/:scrapId.

List endpoints filter to in-scope scraps

GET /api/scraps automatically intersects its org filter with the key's allow-list. A scoped key listing scraps only sees the scraps it is allowed to reach — no 403, just a shorter response:

curl https://api.trawl.me/api/scraps \
  -H "Authorization: Bearer trawl_your_scoped_key"
# → 200 OK — returns only scraps present in scrapIds

Migrating an existing key

scrapIds can only be set at creation — there is no PATCH on API keys. To narrow an existing key, create a new scoped key and revoke the old one from Developers → API Keys.

Rate limits

Only POST /api/scraps/worker/:scrapId is rate-limited on the public API. Read endpoints (GET /api/scraps, GET /api/scraps/:scrapId) are not throttled beyond standard platform protections.

Surface	Limit
POST /api/scraps/worker/:scrapId (per API key)	60 requests / 15 minutes
POST /api/mcp (per-IP, pre-auth — bounds failed-auth floods before any key/token is validated)	60 requests / minute
POST /api/mcp (per caller — all of a user's keys share one bucket)	120 requests / minute

The pre-auth /api/mcp limiter runs before the key/token is checked, so a burst of invalid or guessed credentials is bounded even when it never resolves to an authenticated identity. The per-caller limiter above it takes over once auth succeeds.

Hitting the limit returns 429 Too Many Requests (REST) or JSON-RPC -32005 rate_limited (MCP). Back off and retry after the indicated interval.

See also: MCP Server · Webhooks · Scrap Payload

Next step → Webhooks