Trawl ships two AI-powered features that keep your scraps healthy and your data actionable — without manual intervention.

Auto-fix with AI

Websites change. Selectors break. Instead of babysitting every scrap, let the AI fix it for you.

How it works

When a run fails (error or 0 items), or returns dramatically fewer items than the recent baseline (silent regression detection), Auto-fix sends the broken request script and the error output to the AI, which rewrites the script to adapt to the new page structure. The fixed script is saved and the scrap is re-run automatically.

Enable it in Settings → AI Features → Auto-fix with AI.

Notifications: when Auto-fix and Alert on failure are both enabled, the generic failure email is suppressed in favour of an outcome-aware email (recovered or needs-review). See Alerts with Auto-fix for the full table.

Safety guardrails

Auto-fix is not a runaway loop — it has built-in limits:

Guardrail	Value
Max attempts per hour	6
Consecutive failure breaker	3 (default) — after 3 consecutive failed auto-fix attempts, the feature is disabled for that scrap. Configurable via trawl.autofix.failureThreshold in server config.
Re-enable	Automatic on the next manual successful run (from the UI or API)
Regression detection	Triggers when a run returns fewer than 20 % of the recent baseline item count, based on a minimum baseline of 5 items across at least 3 recent successful runs. Counts as an auto-fix trigger (or a notification when auto-fix is off).

If Auto-fix is disabled by the breaker, you will see a notice in the scrap workspace. Run the scrap manually, fix the issue if needed, and a successful run re-arms auto-fix.

When to use it

• A site updated its HTML and your selectors stopped matching
• A navigation flow changed (new cookie banner, redirects)
• Timeout errors due to changed page load behavior
• A site partially restructured: results dropped to a small fraction of the usual count without an explicit failure

Manual AI Fix

You can also trigger a one-shot AI fix from the workspace toolbar — click the Fix button that appears after a failed run. This does not count against the auto-fix hourly limit.

AI Digest

Raw data is useful, but sometimes you want the highlights. AI Digest sends an AI-generated summary of each scheduled run to your inbox.

Setting it up

1. Open Settings → AI Features → AI Digest and toggle it on.
2. Write a digest prompt — tell the AI what to focus on (max 2,000 characters).
3. Optionally set a recipient email (defaults to the email results recipient).

Prompt examples

Good prompts are specific and action-oriented:

• "Highlight any price drops greater than 10%. List new items not seen in the previous run."
• "Summarize the top 5 headlines. Flag anything related to AI regulation."
• "Compare today's inventory count to yesterday. Alert if any SKU dropped below 50 units."

Test preview

Before waiting for the next cron run, use the Test Digest button to generate a preview based on the latest result. This lets you iterate on the prompt until the output is exactly what you need.

When does the digest run?

The digest runs after each scheduled (cron) execution — same trigger as email results. Manual runs and API-triggered runs do not generate a digest.

---

See also: Schedule · Notifications · Scraping

Next step → Data quality