Trawl

Account Namespace

This is the canonical reference for the TRAWL.account.* runtime namespace exposed to request scripts (the bare account.* global remains available as a legacy alias). It is linked directly from the Trawl web app (Settings → Account). For a hands-on walkthrough, see the Scraping Advanced guide.

What is the account namespace?

When a scrap has credentials configured via Settings → Account in the Trawl web app, the worker decrypts them and exposes them to the request script as VM globals — separate from custom parameters. They are accessible under the TRAWL.account.* sub-namespace (canonical) and also as the bare account.* alias.

Node
TRAWL.account.username          // string — the stored username
TRAWL.account.password          // string — the stored password
TRAWL.account.session           // object|null — persisted session, if any
TRAWL.account.session.cookies   // array of Puppeteer cookies
TRAWL.account.session.savedAt   // Date — when the session was persisted
TRAWL.account.session.expiresAt // Date — when the session will be auto-purged

Each field is also reachable at the bare account.* alias (e.g. account.username).

Fields

Field Type Description
TRAWL.account.username string The stored username (decrypted at run time, encrypted at rest).
TRAWL.account.password string The stored password (decrypted at run time, encrypted at rest).
TRAWL.account.session object | null Present only after a successful saveSession(cookies) call on a previous run. null before the first login or after the session has been cleared / expired.
TRAWL.account.session.cookies Cookie[] Array of Puppeteer cookies to restore with page.setCookie(...).
TRAWL.account.session.savedAt Date Timestamp of the last successful saveSession(cookies) call.
TRAWL.account.session.expiresAt Date Absolute expiry — after this date, the background cron purges TRAWL.account.session but preserves username / password.

Session lifecycle

  1. First run: TRAWL.account.session is null. The script runs a full login flow with TRAWL.account.username / TRAWL.account.password, then calls await saveSession(await page.cookies()) to persist the authenticated cookies.
  2. Subsequent runs: TRAWL.account.session.cookies is available. The script restores cookies with page.setCookie(...) and skips the login flow entirely.
  3. Expiry: when TRAWL.account.session.expiresAt is in the past, a background job resets TRAWL.account.session to null (username / password are preserved). The next run falls back to the full login flow.
  4. Manual clear: Settings → Account → Clear session in the Trawl web app immediately nulls TRAWL.account.session.

Credentials and saved cookies are always encrypted at rest.

History fields — accountUsed / sessionUsed

Every execution written to the scrap history carries two booleans that reflect how the worker ran:

  • accountUsedtrue when the envelope sent to the worker carried TRAWL.account.username / TRAWL.account.password. Set at dispatch time, before the worker responds.
  • sessionUsedtrue when the worker response envelope contained _sessionUsed: true, meaning the request script actually consumed TRAWL.account.session.cookies (skipping the login flow). Set when the worker response is processed.

These two flags are independent: a run can have accountUsed=true with sessionUsed=false (first login, no saved session yet), or accountUsed=true with sessionUsed=true (cookie reuse path).

saveSession(cookies) helper

saveSession is an async helper injected into the request VM, NOT part of the TRAWL.account namespace. It persists the given cookies to TRAWL.account.session encrypted at rest and sets savedAt / expiresAt. Typical usage: call it at the end of a successful login sequence so the next run can reuse the session.

Node
await saveSession(await page.cookies());

Triggering runs with credentials via the API

When triggering a scrap from the public API, credentials are NOT passed on the request body — they always come from the stored Settings → Account data. The API body is strictly reserved for TRAWL.* parameters. See the Parameters reference for the parameter envelope.


See also: Scraping Advanced · Parameters · Data Quality

Next step → Security