Schema drift policies

The three policies#

When an incoming batch no longer matches the destination table, the pipeline's schema-drift policy decides what happens. Three policies exist, and the stored values are the ones in parentheses:

Every pipeline that uses Evolve or Rescue also carries a fallback policy for the cases the primary policy cannot handle. A fallback is always block_and_alert or rescue — never auto_add.

Behaviour matrix#

Non-structural drift — nullability changes and column reordering — always proceeds under every policy. The platform matches columns by name, not by position.

Evolve is additive only. The only DDL it will ever issue is a nullable ADD COLUMN. It does not widen a type, does not tighten or relax a constraint, and does not drop anything. Any non-additive change is handed to the fallback policy.

Schema evolution needs an engine adapter, which exists only for postgres, cockroachdb, mysql, mariadb and singlestore. On any other destination Evolve has nothing to issue and the fallback applies.

The rescue column#

Rescue never blocks a run. Added fields and type-changed fields are written into a reserved column, _abrq_rescued_data:

• JSONB on the PostgreSQL family (postgres, cockroachdb)
• JSON on the MySQL family (mysql, singlestore, mariadb)

The payload is shaped:

{
  "_schema_version": 3,
  "_rescued_at": "2026-07-30T09:14:02Z",
  "_reasons": ["unexpected_column"],
  "fields": {
    "customer_tier": "gold",
    "signup_source": "referral"
  }
}

Rows that did not drift leave _abrq_rescued_data NULL, so a WHERE _abrq_rescued_data IS NOT NULL filter is an exact list of the affected rows and costs nothing on the rest of the table.

Promoting a rescued field#

Rescue is a holding pattern, not a destination. When a rescued field turns out to be real, promote it:

1. List the rescued rows for the pipeline and inspect the distinct field names in the payloads.
2. Promote one field to a real column. In a single destination transaction the platform issues the ALTER TABLE … ADD COLUMN, back-fills the new column from every payload that carries the field, and strips that field out of the remaining payloads.

Properties of the promote operation:

• Forward-only. There is no demote. To undo, drop the column yourself.
• Idempotent. Re-running a promote that already completed is a no-op — the payloads no longer carry the field.
• Type-validated. The SQL type you promote to is checked against a whitelist; an arbitrary type expression is refused rather than interpolated.

Promote one field at a time. A payload with three unexpected fields takes three promotes, each with its own transaction.

CDC is different#

Change data capture mirrors an operator's table, so the rules are stricter.

Warning. Under CDC, destructive drift always blocks — under every policy, including Rescue. The run stops with: destructive source schema change (…) — never auto-applied; resolve the mirror manually, then resume. Reconcile the mirror yourself, then resume the CDC table.

Under Rescue, non-destructive CDC drift is recorded as a drift event but no new schema version is created. The consequence is that the same fields are re-detected on the next sync and re-recorded — the pipeline keeps flowing, and the event count keeps rising until you either evolve the mirror or promote the field.

ETL uses a different vocabulary#

ETL tasks do not use the three-policy model. An ETL task's on_schema_drift field takes block_and_alert or proceed only, and ETL never issues ALTER TABLE against an operator-owned destination table under any setting. See Load modes.

Defaults by pipeline family#

The reasoning is uniform: CDC mirrors an authoritative table and must never diverge silently, while the ingestion families take whatever an upstream system hands them and must not stop a feed over an extra field.

Alerts#

Every drift decision raises an alert:

• Severity critical when the run was blocked.
• Severity warning in every other case — evolved, rescued, proceeded.

Alerts are pushed to every enabled notification channel.

Note. Email feeds have no notification channel. Drift on an email feed is still recorded in the drift registry and still counted in the metrics, but nothing is delivered. Watch the metrics for that family.

Metrics#

Two counters cover the whole surface:

A rising abrq_rescued_rows_total is the cue to promote a field: it means more data is landing in the catch-all than in typed columns. A non-zero rate(abrq_schema_drift_events_total{action="blocked"}[1h]) means a pipeline is stopped and waiting for you.

See Metrics for the full list.