The trawl CLI is the fastest way to manage scraps from your terminal. Install once, authenticate once, then list, run, watch, and debug from anywhere — no browser needed. Pairs well with CI/CD pipelines and the Claude Code skills.

Install

npm install -g @trawlme/cli

Or run ad-hoc without installing:

npx @trawlme/cli scraps list

Authenticate

Interactive (default)

trawl login

Prompts for email and password. Credentials are stored locally for subsequent commands.

CI/CD — environment variable

export TRAWL_TOKEN=<your-jwt>
trawl scraps list

TRAWL_TOKEN overrides the stored session for the current command — no trawl login call needed, and nothing is written to disk. Ideal for GitHub Actions, Vercel, Netlify, and any other non-interactive context.

Requires @trawlme/cli ≥ 1.17.0. On earlier versions, TRAWL_TOKEN is only consumed by trawl login itself (to persist the token) — run trawl login --token <jwt> first, then subsequent commands read the stored session.

CI/CD — token flag

trawl login --token <jwt>

Custom API URL (self-hosted)

trawl login --url https://self-hosted.example.com

Common workflows

List your scraps

trawl scraps list
trawl scraps list --json                        # machine-readable JSON
trawl scraps list --status failure              # filter by last run status
trawl scraps list --status success
trawl scraps list --status never                # scraps that have never run

Get a single scrap

trawl scraps get <id>
trawl scraps get <id> --json

Run a scrap on demand (UI / JWT)

trawl scraps run <id>           # blocks until the run completes
trawl scraps run <id> --watch   # blocks, streaming activity output as it runs

run calls GET /api/scraps/load/:id authenticated with your JWT (interactive login) and blocks for the whole synchronous run (typically 30–250s) — it does not return until the scrap finishes or errors. It is the same code path used by the Trawl web interface and is rate-limited per user. For a non-blocking, fire-and-forget launch, use trigger below.

Trigger a scrap without blocking (CI/CD)

trawl trigger <id>           # async: queues the run, returns immediately
trawl trigger <id> --wait    # synchronous: block until the run completes
trawl trigger <id> --watch   # queue, then stream the activity log until the run ends

trigger calls POST /api/scraps/worker/:id, authenticated the same way as every other CLI command: your session JWT, via TRAWL_TOKEN or a stored trawl login. There is no separate API-key auth path in the CLI today — the difference from run is the endpoint's behavior, not the auth method.

By default trigger is asynchronous — it queues the worker run and returns immediately instead of holding the connection for the whole 30-250s run. For pipelines that must gate on the result, add --wait (block until completion) or --watch (stream until the run ends); both make the command's exit reflect the run's outcome.

CI/CD tip: use trigger (not run) in GitHub Actions, Vercel hooks, and other automated contexts — it won't hold your job's connection open for the whole run. Add --wait or --watch when the step must block on the result.

# GitHub Actions example (requires @trawlme/cli >= 1.17.0)
- name: Trigger scrap
  env:
    TRAWL_TOKEN: ${{ secrets.TRAWL_TOKEN }}
  run: npx @trawlme/cli trigger ${{ vars.SCRAP_ID }} --watch

Watch a run live (SSE stream)

trawl scraps watch <id>

Attaches to the SSE stream for the scrap's next run. Use after run or trigger to follow output in real time.

Get recent results

trawl scraps data <id>          # latest run payload
trawl scraps data <id> --json   # raw JSON

Debug a failing scrap

When a scrap returns errors, empty results, or has been auto-fixed, use the diagnostic commands to understand what happened without opening the web UI.

Quick diagnosis — last run status + failed selector + autofix summary:

trawl scraps doctor <id>
trawl scraps doctor <id> --json   # raw run + autofix JSON

doctor fetches the last run and prints: status badge, error message, failed CSS selector, block detection status, proxy tier (Tier 0–4 abstract label), empty-context page URL/anchor count, and a one-line autofix summary if the engine ran.

Cost, proxy vendor, and egress IP are not exposed — by design.

List past runs + inspect a specific one (any run, not just the last):

trawl scraps history <id>            # table of recent runs: status, time, proxy tier, failureKind
trawl scraps history <id> --json     # newest first; --limit <n> caps rows (default 20)
trawl scraps run-info <hid>          # one run's detail by its history id (status, tier, failureKind, error + selector)
trawl scraps run-info <hid> --json

doctor/autofix inspect the last run; history + run-info reach any run. List runs with history, take a run's hid, then pass it to run-info for the full diagnosis. run-info returns the proxy tier + failureKind to the scrap owner via GET /api/historys/:hid.

Full autofix detail — decision, diff, dry-run results, knowledge used:

trawl scraps autofix <id>
trawl scraps autofix <id> --json

autofix shows the fix decision (applied / failed / skipped), classification, the unified diff of the code change, dry-run pass rate, and the knowledge fingerprints the engine consulted.

Failure diagnostics inline with data:

trawl scraps data <id> --errors   # diagnosis when the last run failed; item count otherwise

Download the captured page HTML (to inspect what the worker actually saw):

trawl scraps snapshot <id>                     # captured page at end of run
trawl scraps snapshot <id> --error             # error-path snapshot (selector failure context)
trawl scraps snapshot <id> --error -o page.html  # write to file instead of stdout

Typical debug flow:

trawl scraps doctor <id>           # 1. what went wrong + autofix outcome
trawl scraps autofix <id>          # 2. if autofix ran — inspect the diff + dry-run
trawl scraps snapshot <id> --error # 3. inspect the DOM the worker saw when it failed
trawl scraps run <id> --watch      # 4. trigger a re-run after fixing the script

Create or update a scrap

trawl scraps create -t "HN front page" -u https://news.ycombinator.com
trawl scraps update <id> --cron "0 * * * *" --alert me@example.com
trawl scraps update <id> --no-cron   # disable schedule

Manage account credentials (for auth-gated scraps)

trawl scraps account set <id> -u username -p password
trawl scraps account status <id>
trawl scraps account clear-session <id>  # force a fresh login next run
trawl scraps account delete <id>

See Account & sessions for the full auth-gated scraping workflow.

Delete a scrap

trawl scraps rm <id>
trawl scraps rm <id> --force   # skip confirmation prompt

Install Claude Code skills

The CLI bundles the @trawlme/skills package. Install all Trawl skills into Claude Code in one step:

trawl skills install          # → ~/.claude/skills/ (user-level, all projects)
trawl skills install --local  # → .claude/skills/ (project-level)
trawl skills list             # show bundled skills and install status
trawl skills update           # reinstall to pick up CLI version bumps

See Skills for what each skill does.

CI/CD usage

Use trigger (not run) in automated contexts — it's async by default and won't hold your job's connection open for the whole run. Both run and trigger authenticate the same way: a session JWT, supplied via TRAWL_TOKEN (env var — requires @trawlme/cli ≥ 1.17.0) or trawl login --token <jwt>. There is no separate API-key auth path in the CLI today.

Because it's a session JWT, it expires after 7 days — plan to refresh the stored secret periodically, e.g. by running trawl login && trawl token and updating it.

GitHub Actions

- name: Trigger scrap
  env:
    TRAWL_TOKEN: ${{ secrets.TRAWL_TOKEN }}
  run: |
    npx @trawlme/cli trigger ${{ vars.SCRAP_ID }} --watch

Store a session JWT (from trawl login, retrieved with trawl token) as a repository secret named TRAWL_TOKEN. The --watch flag blocks until the run completes, letting you use the exit code for gating.

Vercel / Netlify

Set TRAWL_TOKEN (a session JWT, obtained via trawl login + trawl token) in environment variables, then call npx @trawlme/cli trigger <id> in your build or deploy hooks.

Environment variables

Variable	Description
TRAWL_TOKEN	Session JWT — overrides the stored session for the current command, no trawl login call needed (requires @trawlme/cli ≥ 1.17.0). 7-day expiry, same as any trawl login session.
TRAWL_API_URL	Override the default API URL (https://api.trawl.me)

Command reference

Command	Description
trawl login [--url] [--token] [--email] [--password]	Authenticate
trawl logout	Remove stored credentials
trawl scraps list [--json] [--status]	List scraps
trawl scraps get <id> [--json]	Fetch one scrap
trawl scraps create -t <title> [-u <url>] [-r <request>] [-d <description>] [--tier <tier>]	Create scrap
trawl scraps update <id> [options]	Update scrap
trawl scraps run <id> [--watch]	Run via JWT (UI path, GET /api/scraps/load/:id) — blocks until the run completes
trawl trigger <id> [--wait] [--watch]	Trigger via JWT (TRAWL_TOKEN or trawl login), CI/CD path, POST /api/scraps/worker/:id; async by default, --wait blocks until done. (trawl scraps trigger <id> still resolves — hidden legacy alias.)
trawl scraps watch <id>	Stream live output
trawl scraps data <id> [--json] [--errors]	Get latest results; --errors shows failure diagnostics instead when the last run failed
trawl scraps history <id> [--json] [--limit <n>]	List a scrap's past runs (status, time, tier, failureKind), newest first
trawl scraps run-info <hid> [--json]	One run's detail by its history id (status, tier, failureKind, error + selector)
trawl scraps doctor <id> [--json] [--autofix]	Diagnose last run: error, failed selector, block status, proxy tier, autofix outcome
trawl scraps autofix <id> [--json]	Full autofix detail: decision, diff, dry-run results, knowledge used
trawl scraps snapshot <id> [--error] [-o <file>]	Download captured page HTML; --error fetches the error-path snapshot
trawl scraps rm <id> [--force]	Delete scrap
trawl scraps account set <id>	Set account credentials
trawl scraps account status <id>	Show account + session state
trawl scraps account clear-session <id>	Clear cached session
trawl scraps account delete <id>	Remove account credentials
trawl skills list	List bundled Claude Code skills
trawl skills install [skill] [--local]	Install one or all skills
trawl skills uninstall [skill] [--local]	Remove skills
trawl skills update [skill] [--local]	Reinstall (sync with CLI version)

---

See also: API keys · Skills · Using Trawl with Claude

Next step → Skills