This is the canonical reference for data validation on scrap results. It is linked directly from the Trawl web app (Settings → Data Validation). For a hands-on walkthrough, see the Scraping Advanced guide.

What gets validated

After a successful run (result.type === 'success'), the scrap's stored JSON Schema — if any — is compiled and run against the worker result payload. If validation fails, the run is downgraded to validation_failed (a distinct history status) and a history activity is recorded.

Validation runs on every execution — manual, scheduled and API-triggered. Runs that already failed upstream (error, 0 items) are not re-validated.

Defining a schema — checkSchema

The schema is stored on the scrap as checkSchema: a plain JSON Schema object (draft 2020-12, compiled by the JSON Schema validator at save time and on every run).

Edit it in Settings → Data Validation → Check data return in the Trawl web app. Paste or hand-write a schema describing the expected shape of the result array returned by returnData(result).

A typical schema for an array of items with title, price, url:

{
  "type": "array",
  "items": {
    "type": "object",
    "required": ["title", "price", "url"],
    "properties": {
      "title": { "type": "string", "minLength": 1 },
      "price": { "type": "string" },
      "url":   { "type": "string", "format": "uri" }
    }
  }
}

Saving an invalid schema (one that fails to compile) returns 422 VALIDATION_ERROR from the API with the compiler's error message. Setting checkSchema to null disables validation entirely for that scrap.

Inferring a schema from an example

The API accepts a shortcut: if you submit a check array (one or more example result objects), Trawl infers a matching schema from the example structure and stores it as checkSchema. This is the same inference helper used to auto-migrate legacy scraps (see below).

The validation_failed history status

When the schema check fails, the run lands in history with type: 'validation_failed' — not success and not error. This lets dashboards and alerts distinguish three outcomes:

Status	Meaning
success	Run succeeded, result passed (or no schema configured)
validation_failed	Run succeeded, result violated the schema
error	Run threw before returning data

A dedicated activity entry is created on every validation_failed run with the compiler's first N errors attached, so users can see which fields broke the shape.

Alerts on validation failures

Enabling Alert in Settings → Automation sends an email when the scrap transitions to validation_failed, just like error / empty transitions. The validation_failed status participates in the same state machine as the other failure types.

Legacy check[] mode — deprecated

Before the JSON Schema migration, scraps stored a literal example array under a field called check. That legacy mode is deprecated:

• On save, the API auto-converts a supplied check[] payload into a checkSchema via shape inference. A warning is logged and the scrap now uses checkSchema for all future validation passes.
• Reading a scrap with a stored check[] still returns the legacy field for backwards compatibility, but new writes should target checkSchema directly.
• The legacy path will be removed in a future release — update any API clients to send checkSchema (or a check[] example you are happy to have inferred).

---

See also: Scraping Advanced · Notifications · AI Features

Next step → Results & payload