Trawl

Account Namespace

This is the canonical reference for the TRAWL.account.* runtime namespace exposed to request scripts (the bare account.* global remains available as a legacy alias). It is linked directly from the Trawl web app (Settings → Account). For a hands-on walkthrough, see the Scraping Advanced guide.

What is the account namespace?

When a scrap has credentials or a captured session configured via Settings → Account in the Trawl web app (or the CLI's session capture / session set), the worker decrypts them and exposes them to the request script as VM globals — separate from custom parameters. They are accessible under the TRAWL.account.* sub-namespace (canonical) and also as the bare account.* alias.

Node
TRAWL.account.username          // string — the stored username
TRAWL.account.password          // string — the stored password
TRAWL.account.session           // object — the stored session (see below)
TRAWL.account.session.cookies   // array of Puppeteer-shaped cookies
TRAWL.account.session.origins   // array of { origin, localStorage: [{name, value}] } — per-origin local storage
TRAWL.account.session.savedAt   // Date | null — when the session was last persisted
TRAWL.account.session.expiresAt // Date | null — when the session will be auto-purged

Each field is also reachable at the bare account.* alias (e.g. account.username).

A session that hasn't been captured yet is empty arrays, not null. TRAWL.account.session is always an object once an account is configured — cookies and origins default to [] rather than the field itself being absent. That means if (TRAWL.account?.session?.cookies) is always truthy (an empty array is truthy in JS) — check TRAWL.account.session.cookies.length instead if your script needs to branch on "is there actually a saved session". See the Scraping Advanced guide for the recommended pattern.

Fields

Field Type Description
TRAWL.account.username string The stored username (decrypted at run time, encrypted at rest).
TRAWL.account.password string The stored password (decrypted at run time, encrypted at rest).
TRAWL.account.session object Always present once an account is configured. cookies and origins are [] before the first captured session.
TRAWL.account.session.cookies Cookie[] Puppeteer-shaped cookies, replayed automatically by the worker (see below) — scripts don't need to restore them.
TRAWL.account.session.origins Origin[] { origin, localStorage: [{name, value}] } entries, replayed automatically per origin. [] on a session captured before this field existed, or with no localStorage in scope.
TRAWL.account.session.savedAt Date | null Timestamp of the last saved session.
TRAWL.account.session.expiresAt Date | null Absolute expiry — after this date, a background job clears the stored session (cookies and origins come back as [] on the next run) but preserves username / password.

Automatic session replay

The worker replays a stored session before your script's first navigation — this is new behavior, and it changes what a script needs to do:

  • Cookies are applied to the browser context (not just one page) before the script runs, so every page the script opens afterward already carries them.
  • Local storage is seeded per origin, also before the script's first navigation, and re-seeded for any additional page or popup the script opens itself (a login flow that spawns an OAuth popup, for example).

Your script no longer needs to restore cookies onto the page itself — the page already has the session by the time your script's first line runs. The old per-page cookie-restore call this used to rely on is deprecated Puppeteer API surface besides (cookies moved to the browser-context level), so new scripts should not call it at all. If an older script still does, it's a harmless duplicate of what the worker already applied.

What a script's job actually is: detect whether the session is already logged in, and only run the credential login flow if it isn't. See the Scraping Advanced guide for that pattern.

Getting a session onto a scrap

Three ways, in order of how much they do for you:

  1. trawl scraps account session capture <id> (recommended) — opens a real, visible Chrome window at the scrap's target URL. Log in there exactly as you normally would, 2FA included, then press Enter back in the terminal. The CLI reads the resulting session over the Chrome DevTools Protocol — cookies and per-origin local storage — and uploads it. Requires a local interactive terminal with a display (it does not work headless, in CI, or over a plain SSH session).

    You stay authenticated as yourself throughout — Trawl never sees your credentials, only the resulting session. Responsibility for lawful use of that session stays with you; this isn't legal advice.

  2. Settings → Account in the Trawl web app.

  3. trawl scraps account session set <id> -c <file> — uploads a session you already have: either a bare Puppeteer cookie JSON array, or a { cookies, origins } storageState-shaped file (the same shape session capture produces).

See the CLI guide for the full command reference.

Session shape and validation

A session uploaded via the API (either CLI path, or Settings → Account) is validated before it's stored — a malformed cookie is rejected with a clear error (422) rather than silently failing to apply at replay time:

Rule Detail
domain Required on every cookie.
expires Unix seconds. A value of -1 ("session cookie" in some export formats) is dropped rather than stored literally.
sameSite Recased to Strict / Lax / None; unspecified is omitted rather than guessed.
sameSite: 'None' Requires secure: true — a browser silently drops such a cookie otherwise, so this is rejected at ingest instead of failing invisibly later.
Unknown cookie fields Dropped silently — only the fields a real cookie needs are kept.
cookies Must be a non-empty array.
origins[].origin Must be a bare origin (scheme://host[:port], no path or query).
origins[].localStorage Array of { name, value } string pairs.

saveSession(cookies) helper

saveSession is an async helper injected into the request VM, NOT part of the TRAWL.account namespace. It persists the given cookies to TRAWL.account.session, encrypted at rest, and refreshes savedAt / expiresAt. It still accepts cookies only — call it at the end of a login sequence your script itself performed:

Node
await saveSession(await page.cookies());

This is a cookies-only refresh: it does not touch or clear any local storage previously captured via session capture/Settings → Account. In contrast, uploading a session via the CLI or the web app fully replaces the stored session each time — a bare cookie-array re-upload there clears any previously-stored local storage. Use saveSession() from a script for a routine cookie refresh; use session capture or session set when you also need local storage in the mix.

Session lifecycle

  1. First run: TRAWL.account.session.cookies is []. The script runs a full login flow with TRAWL.account.username / TRAWL.account.password, then calls await saveSession(await page.cookies()) to persist the authenticated cookies — or a session is captured ahead of time via session capture/Settings → Account, in which case cookies and local storage are already there on the first run.
  2. Subsequent runs: the worker replays the stored session automatically (see above) before the script runs.
  3. Expiry: when TRAWL.account.session.expiresAt is in the past, a background job clears the stored session — cookies and origins come back as [] on the next run (username / password are preserved). The next run falls back to the full login flow.
  4. Manual clear: Settings → Account → Clear session in the Trawl web app immediately clears TRAWL.account.session.

Credentials and saved sessions are always encrypted at rest.

History fields — accountUsed / sessionUsed

Every execution written to the scrap history carries two booleans that reflect how the worker ran:

  • accountUsed — true when the envelope sent to the worker carried TRAWL.account.username / TRAWL.account.password. Set at dispatch time, before the worker responds.
  • sessionUsed — true when the worker confirms at least one stored cookie was actually applied during automatic replay, or the script itself restored at least one cookie onto the page directly (the legacy pattern — harmless if it re-applies what the worker already set). Set when the worker response is processed.

These two flags are independent: a run can have accountUsed=true with sessionUsed=false (first login, no saved session yet), accountUsed=true with sessionUsed=true (session reuse path), or accountUsed=false with sessionUsed=true — a session-only account (captured via session capture, session set or Settings → Account, with no username/password stored) whose session was replayed. TRAWL.account.username / password are null on that run.

Not a guaranteed fix

A valid, freshly-captured session is the right first move on a site that's asking you to log in — but it isn't a guaranteed fix. Some sites also flag a session replayed from a datacenter IP, independent of whether the session itself is genuinely good — a residential proxy tier is the lever there, not the session.

Triggering runs with credentials via the API

When triggering a scrap from the public API, credentials are NOT passed on the request body — they always come from the stored Settings → Account data. The API body is strictly reserved for TRAWL.* parameters. See the Parameters reference for the parameter envelope.


See also: Scraping Advanced · Parameters · Data Quality · AI Features

Next step → Security