Backup and Restore

What a metadata backup contains#

A backup captures the framework database's configuration and progress state as a single file. The table set is derived from the live ORM schema — every configuration and state table is included automatically — minus an explicit denylist of transient execution history. In practice:

Included: connectors, CDC table definitions and their checkpoints, the full ETL surface (projects, environments, tasks, Jobs, schedules) including etl_task_watermarks, streams and their destinations, file ingestions, exports and their watermarks, governance configuration (masking, retention), notification and provider settings, users, roles, and the encrypted secrets table.

Excluded (transient): run history rows and run logs across all families, run metrics, quarantine rows, connection-test results, the JWT revocation list, the audit log, and the backup subsystem's own bookkeeping.

Because checkpoints and watermarks are included, a restored deployment resumes incremental pipelines where they left off — no full re-snapshot.

Warning. A metadata backup does not cover the datastore — the Postgres database holding CDC mirror tables and landed data — nor any data written to your destinations. That is data, not metadata. Back up the datastore (and your destinations) with your own database backup tooling on whatever cadence your recovery objectives require.

Always encrypted#

Every backup — CLI dump, UI download, scheduled auto-backup — is sealed as a Fernet-encrypted envelope with the deployment's master key (ABRQ_MASTER_KEY). There is no plaintext export path. Consequently:

• Restoring requires an instance configured with the same master key.
• Name backup files <NAME>.json.enc so operators expect ciphertext.
• The envelope records the schema (migration) version at dump time; restore refuses a mismatched schema unless explicitly overridden.

Manual backup and restore (CLI)#

On a Docker Compose deployment:

docker compose exec backend uv run python scripts/backup_metadata.py \
  dump --out /var/abrq-dip/backup-<DATE>.json.enc

docker compose cp backend:/var/abrq-dip/backup-<DATE>.json.enc ./

Store the copy outside the deployment directory — ideally on a different disk or host.

Restore:

docker compose cp backup-<DATE>.json.enc backend:/tmp/restore.json.enc
docker compose exec backend uv run python scripts/backup_metadata.py \
  restore --in /tmp/restore.json.enc

Restore semantics:

• Default: abort on non-empty tables. Without flags, the restore refuses to touch any target table that already has rows — the safe default against an accidental restore.
• --force — TRUNCATEs every target table first, then inserts from the backup. This is the flag you want during deliberate disaster recovery, and it is destructive.
• --allow-schema-mismatch — proceeds even when the backup's schema version differs from the live database. You almost never want this; migrate the database to the matching version instead.

On Kubernetes, run the same python scripts/backup_metadata.py command inside a running api pod (for example with kubectl exec).

Manual backup and restore (UI)#

Settings → License & Backup carries a Backup & restore card:

• Download backup streams the encrypted dump as abrq-dip-backup-<SCHEMA>-<TIMESTAMP>.json.enc.
• Restore from file uploads a backup with the same semantics as the CLI: a force checkbox (guarded by typing RESTORE) and an allow schema mismatch checkbox. The result panel lists per-table restored row counts — read it to confirm the restore covered what you expected.

Both actions are admin-only and audited.

Scheduled automatic backups#

The Automatic backups card on the same tab manages an opt-in daily job (a singleton setting, off by default):

• When enabled, a daily scheduled task writes an encrypted copy of the metadata backup to a subdirectory under the server's backup root — but only when the configuration changed since the last copy (change-detection by content hash). Enabling, or saving while enabled, writes the first copy immediately; Run check now triggers the same check on demand.
• The backup directory is a subdirectory name under the backup root — jailed to that root, with no absolute paths and no ...

The card shows the last status:

Where the files land, per deployment path#

• Docker Compose (production file): the backup root is bind-mounted from the host (ABRQ_BACKUP_HOST_DIR, default ./backups), so copies are durable on the host filesystem.
• Kubernetes (Helm): set backups.enabled=true to mount a shared backups PVC at the backup root. The claim defaults to ReadWriteMany access so multiple replicas can share it; you can pass an existingClaim. With backups.enabled=false (the default) the chart sets ABRQ_BACKUP_DURABLE=false, and successful auto-backups report ok_ephemeral — the copy exists inside the pod but dies with it.

Warning. Treat ok_ephemeral as "not backed up" for disaster-recovery purposes. Either enable the backups PVC (Kubernetes) or keep the host bind-mount (Compose), and copy backups off the host on your own schedule.

Restore runbook#

To recover a deployment (or roll back after a failed upgrade), restore in this order:

1. Stop the stack (docker compose down — never down -v, which destroys volumes).

2. Provision a clean framework DB at the matching schema: start the database and the migrator, and confirm the migrator exits successfully.

3. Start the backend service.

4. Copy the backup file into the container and restore with --force (deliberate recovery truncates first):

   docker compose cp backup-<DATE>.json.enc backend:/tmp/restore.json.enc
   docker compose exec backend uv run python scripts/backup_metadata.py \
     restore --in /tmp/restore.json.enc --force
   

5. Bring up the rest of the stack (workers, scheduler, frontend) and verify: sign in, check connectors and pipelines, and confirm incremental pipelines resume from their restored checkpoints.

The same master key must be configured before step 4 — a restore on an instance with a different ABRQ_MASTER_KEY cannot decrypt the envelope. For the full upgrade-and-rollback flow around this runbook, see Upgrade and rollback; for key handling, see Secrets and encryption.