Receive real-time HTTP notifications when scraps change state — no polling required.

Note. If you have a webhook that was created before webhooks were generally available, it may be set to active: false. Review your endpoints under Developers → Webhooks and flip active: true to opt in. New webhooks default to active: true.

Create a webhook

From the Trawl web app, open Developers → Webhooks. Click Create, enter a public HTTPS URL, and select the events to subscribe to. Copy the secret immediately — it is shown only once and is required to verify signatures.

Supported events

Event	Triggered when
scrap.success	A scrap run completes successfully
scrap.failure	A scrap run fails

Scoping webhooks to specific scraps

By default, a webhook fires for every scrap in the owning organization. You can optionally restrict a webhook to a subset of scraps by passing a scrapIds array at create or update time.

scrapIds value	Dispatch behaviour
[] (or omitted)	Organization-wide — fires for every scrap. Default. Fully back-compatible with every pre-existing webhook.
["<scrapId>", …]	Restricted — only runs of the listed scraps trigger a delivery. Other scraps are silently skipped for this webhook.

Each entry must be a 24-character hex Mongo ObjectId belonging to the caller's organization. Foreign or unknown ids are rejected at create/update time.

Create a scoped webhook

curl -X POST https://api.trawl.me/api/developers/webhooks \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/trawl-hook",
    "events": ["scrap.success", "scrap.failure"],
    "scrapIds": ["64a1f2e3b5c6d7e8f9a0b1c2", "64a1f2e3b5c6d7e8f9a0b1c3"]
  }'

The response carries the newly-created webhook plus plainSecret (shown once). Runs of any other scrap in the organization will not trigger this webhook.

Update the scope

PUT /api/developers/webhooks/:webhookId accepts a partial body. Sending scrapIds overwrites the current scope — send [] explicitly to make a previously-scoped webhook organization-wide again:

# Narrow an existing webhook to a single scrap
curl -X PUT https://api.trawl.me/api/developers/webhooks/<webhookId> \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{ "scrapIds": ["64a1f2e3b5c6d7e8f9a0b1c2"] }'

# Clear the scope — back to org-wide
curl -X PUT https://api.trawl.me/api/developers/webhooks/<webhookId> \
  -H "Authorization: Bearer <jwt>" \
  -H "Content-Type: application/json" \
  -d '{ "scrapIds": [] }'

Omitting scrapIds from the update body leaves the current scope untouched — only an explicit value (including []) mutates it.

Payload

Every delivery sends a JSON body with the following shape (built from the completed history row):

{
  "historyId": "64a1f2e3b5c6d7e8f9a0b1c2",
  "scrapId": "64a1f2e3b5c6d7e8f9a0b1c3",
  "status": "success",
  "statusDetail": "success",
  "timestamp": "2026-04-17T10:30:00.000Z",
  "runtime": 2384,
  "resultPreview": "[{\"title\":\"Example item\",\"price\":\"19.90€\"}]",
  "triggeredBy": "cron"
}

Field	Type	Description
historyId	string	Id of the persisted history row for the run. Use it to fetch the full result via GET /api/historys/:historyId.
scrapId	string	Id of the scrap that produced the run.
status	"success" | "failure"	Coarse-grained outcome. Mirrors the event name (scrap.success or scrap.failure).
statusDetail	string | null	Finer-grained state when available: "success", "empty", "error", …
timestamp	ISO-8601 string	Delivery time (set on dispatch, not worker-completion time).
runtime	integer | null	Worker roundtrip duration in milliseconds.
resultPreview	string | null	Truncated JSON preview of request.data (up to 1024 bytes, trailing … marker when truncated). null when the run returned no data.
triggeredBy	"user" | "api" | "cron" | null	Source that started the run.

The full event name (scrap.success / scrap.failure) is exposed in the X-Webhook-Event request header — the payload itself intentionally omits it so consumers can log the body verbatim without duplication.

Note on status vs statusDetail: The webhook payload status field is a string — "success" or "failure" — mirroring the event name. It is not the HistoryRun status boolean from the REST/MCP responses. Use statusDetail ("success", "error", "empty", "regression") for fine-grained filtering (e.g., ignore empty runs, alert only on error).

Safety contract

The payload is built from an explicit allow-list. These fields are never emitted:

• Encrypted account ciphertext (scrap.account)
• Raw session cookies
• Environment variables
• Free-form error stacks (status detail is a short enum, error messages are truncated)

The resultPreview blob is passed through a best-effort secret redactor as defence-in-depth on top of the allow-list.

Verify signature

Each request includes an X-Webhook-Signature header — HMAC-SHA256 of the raw body signed with your secret. Always verify before processing:

import crypto from 'crypto';

function verifySignature(rawBody, signature, secret) {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(rawBody)
    .digest('hex');
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expected),
  );
}

// Express example
app.post('/webhook', express.raw({ type: 'application/json' }), (req, res) => {
  const sig = req.headers['x-webhook-signature'];
  const event = req.headers['x-webhook-event']; // 'scrap.success' | 'scrap.failure'
  if (!verifySignature(req.body, sig, process.env.WEBHOOK_SECRET)) {
    return res.status(401).send('Invalid signature');
  }
  const payload = JSON.parse(req.body);
  // handle event + payload.historyId / payload.scrapId / payload.status ...
  res.sendStatus(200);
});

Retry policy

If your endpoint returns a non-2xx status or times out (10 s), delivery is retried automatically. Up to three attempts per event:

• Attempt 1 — immediate.
• Attempt 2 — 1 minute after the first failure.
• Attempt 3 — 5 minutes after the second failure.

After the third failure the delivery is marked permanently failed and no further retries are scheduled. A background sweep runs every 60 seconds to redeliver any pending-retry entries whose nextRetryAt has elapsed — deliveries attached to a webhook that has since been deactivated are skipped.

View all delivery attempts and their responses under Developers → Webhooks → Deliveries.

Full programmatic flow

Trigger a run and receive the result without touching the UI:

1. Get an API key — from Developers → API Keys, create a key (see API Access).

2. Register a webhook — create a webhook subscribed to scrap.success and scrap.failure.

3. Trigger a run

curl -X POST https://api.trawl.me/api/scraps/worker/64a1f2e3b5c6d7e8f9a0b1c2 \
  -H "Authorization: Bearer trawl_your_key_here"

4. Receive the webhook — your endpoint receives a scrap.success (or scrap.failure) payload.

5. Fetch results

curl https://api.trawl.me/api/scraps/64a1f2e3b5c6d7e8f9a0b1c2 \
  -H "Authorization: Bearer trawl_your_key_here"

See also: API Keys · Scrap Payload · Notifications

Next step → CLI