Trawl

Parameters

This is the canonical reference for the TRAWL.* runtime namespace exposed to request scripts. It is linked directly from the Trawl web app (Settings → Parameters). For walkthroughs with code examples, see the Scraping and Scraping Advanced guides.

The TRAWL.* namespace

Inside a request script, Trawl injects a TRAWL object that carries every variable the run can use. Keys are resolved in this order at dispatch time:

  1. Reserved TRAWL.url — always taken from the scrap's url setting. Cannot be overridden by a request body or by a custom parameter with the same name.
  2. Custom parameters (TRAWL.<name>) — every key defined in Settings → Parameters becomes TRAWL.<name>.
  3. API body overrides — when a scrap is triggered via POST /api/scraps/worker/{scrapId} with a JSON body, each top-level key becomes TRAWL.<key>. TRAWL.url is the single exception — it is merged last from the stored scrap setting and cannot be overridden.

The legacy @variable syntax (e.g. @url, @query) is no longer supported. Only TRAWL.* keys are honoured (removed in #799).

Reserved keys

Key Source Notes
TRAWL.url Scrap setting (scrap.url) Mandatory. Always injected from the scrap's url field. Cannot be overridden by the API body or by a parameter with the same name.

Any other key starting with TRAWL. but not explicitly declared in Settings → Parameters is allowed — it simply remains undefined unless supplied via the API body.

Custom parameters — TRAWL.<name>

Custom parameters are defined in Settings → Parameters and exposed as TRAWL.<name> globals. Each parameter set can be a plain object of static values or use the template helpers below.

Multiple parameter sets may be declared; for cron-scheduled runs, one set is picked at random. Manual runs from the web app use the last-saved parameter set.

JSON
[
  { "query": "javascript frameworks", "limit": 20 },
  { "query": "rust webassembly",      "limit": 20 }
]

Inside the request:

Node
await page.goto(`https://news.ycombinator.com/search?q=${TRAWL.query}&l=${TRAWL.limit}`);

Template functions — DATE / RANDOM

Parameter values can use two template helpers that are evaluated at dispatch time:

Function Signature Example Resolves to
DATE DATE(delta, unit, format) DATE(-1, days, YYYY-MM-DD) Yesterday formatted as YYYY-MM-DD
RANDOM RANDOM(a, b, c, ...) RANDOM(foo, bar, baz) One value picked at random from the list

Both helpers resolve before the worker dispatch — the request script only ever sees the final string.

Account credentials live under TRAWL.account.*

Account credentials are exposed under the TRAWL.account.* sub-namespace (TRAWL.account.username, TRAWL.account.password, TRAWL.account.session), built server-side from your encrypted credentials. The legacy bare account global remains available as an alias. They are NOT custom parameters — a caller cannot set them via the request body (the worker strips any caller-supplied account / url key). See the Account Namespace reference for the full contract.

API body envelope

When triggering a scrap externally:

cURL
curl -X POST https://api.trawl.me/api/scraps/worker/<scrapId> \
  -H "Authorization: Bearer <accessToken>" \
  -H "Content-Type: application/json" \
  -d '{ "query": "machine learning", "date": "2026-01-01" }'

Every top-level key in the body becomes TRAWL.<key>. TRAWL.url is always sourced from the scrap's stored url and can never be overridden.


See also: Scraping Advanced · Account Namespace · API Keys

Next step → Account & sessions