CDC strategies

The enforced matrix#

A CDC table pairs a strategy with a source engine. Only four combinations exist:

Nothing else works. mariadb, cockroachdb, singlestore, snowflake and every non-relational engine have no change-capture path at all.

Warning. The strategy picker in the UI offers wider combinations than the platform supports, and an unsupported pairing saves without complaint. It fails on the first sync, with: strategy '<name>' does not support engine '<engine>'. Check the pairing against the table above before you register a CDC table.

CDC mirrors can only be written to postgres or mysql destinations — see Connectors.

timestamp#

Detection. A polling strategy. Each sync selects rows whose configured timestamp column is greater than the stored watermark, then advances the watermark to the highest value it read. It needs no database-level configuration, no elevated grants, and no restart — which is why it is the strategy to reach for when you cannot change the source server.

Delete detection. timestamp cannot detect hard deletes. A row removed with DELETE simply stops appearing in the source and remains in the mirror forever. If the application soft-deletes, configure the soft-delete column and the strategy will treat a flagged row as a delete. If it hard-deletes, use a log-based strategy or accept that the mirror is insert/update-only.

Tuning. The timestamp column, an optional soft-delete column, the poll interval, and the batch size. Choose a column the source updates on every write — a created_at column silently misses updates.

Prerequisites. SELECT on the source table, and an indexed timestamp column. Without an index every poll is a full scan.

log_pg (PostgreSQL logical decoding)#

Detection. Reads PostgreSQL's write-ahead log through logical decoding, so inserts, updates and deletes are all observed as they were committed — no polling window, no dependence on an application-maintained column.

Delete detection. Yes, natively. Hard deletes are captured.

Tuning. Poll interval and batch size govern how often the slot is drained and how much is taken per drain. The replication slot is named abrq_<datasource>_<schema>_<table> and the bundled test_decoding output plugin is used, so no third-party plugin has to be installed on the server.

Source prerequisites.

1. wal_level = logical in postgresql.conf. This requires a server restart — it cannot be changed with a reload.
2. A role with the REPLICATION attribute, plus SELECT on the captured table.
3. Enough max_replication_slots headroom for one slot per CDC table.

ALTER SYSTEM SET wal_level = 'logical';
-- restart the server, then:
SHOW wal_level;

CREATE ROLE <ABRQ_CDC_USER> WITH LOGIN REPLICATION PASSWORD '<PASSWORD>';
GRANT SELECT ON <SCHEMA>.<TABLE> TO <ABRQ_CDC_USER>;

Warning. A replication slot that is never drained holds WAL on the source server indefinitely and will eventually exhaust its disk. If you retire a CDC table, delete it in Abrq DIP so the slot is released — or drop the slot manually.

log_mysql (binary log)#

Detection. Reads the MySQL binary log as a replica would, so every committed insert, update and delete is captured in commit order.

Delete detection. Yes, natively.

Tuning. Poll interval, batch size, and the replication server_id the platform presents to the source. The default is 21001; it must be unique across everything reading that server's binary log, including real replicas and any second Abrq DIP instance.

Source prerequisites.

1. log_bin = ON.
2. binlog_format = ROW. Statement or mixed format does not carry row images.
3. binlog_row_metadata = FULL. MySQL 8.0.14 and later default to MINIMAL, which omits the column names the platform needs to map rows.
4. Grants: REPLICATION CLIENT and REPLICATION SLAVE, plus SELECT on the captured table.
5. A server_id not used by any other replica.

SHOW VARIABLES WHERE Variable_name IN
  ('log_bin', 'binlog_format', 'binlog_row_metadata', 'binlog_row_image');

GRANT REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO '<ABRQ_CDC_USER>'@'%';
GRANT SELECT ON <DATABASE>.<TABLE> TO '<ABRQ_CDC_USER>'@'%';

Binary-log retention on the source must exceed the longest expected outage of the CDC worker. If the log segment holding the last read position is purged, the table has to be re-seeded.

log_mssql (SQL Server CDC)#

Detection. Reads SQL Server's own CDC shadow tables through the cdc.fn_cdc_get_all_changes_* functions. SQL Server captures the changes; Abrq DIP consumes them.

Delete detection. Yes, natively.

Tuning. Poll interval and batch size. The retention of the shadow tables is controlled by SQL Server's CDC cleanup job, not by Abrq DIP.

Source prerequisites.

1. SQL Server Agent must be running — the capture and cleanup jobs depend on it.
2. Enable CDC once per database:

EXEC sys.sp_cdc_enable_db;

3. Enable CDC per captured table:

EXEC sys.sp_cdc_enable_table
  @source_schema = N'<SCHEMA>',
  @source_name   = N'<TABLE>',
  @role_name     = NULL;

4. Grants for the Abrq DIP login: SELECT on the shadow table cdc.<schema>_<table>_CT, and EXECUTE on the matching cdc.fn_cdc_get_all_changes_* function.

Note. The connection test probes MySQL binary-log settings, but it cannot verify PostgreSQL wal_level and it cannot verify that SQL Server CDC has been enabled on the database or the table. Check those yourself with the statements above before the first sync — otherwise the first run is where you find out.

Choosing a strategy#

Schema drift on a CDC mirror is governed separately, and destructive drift always blocks — see Schema drift policies.