Connectors

The engine set#

Abrq DIP ships 14 connector engines. A connector is always an engine plus credentials; two connectors on the same engine have identical capabilities.

postgres, mysql, mariadb, cockroachdb, sqlserver, singlestore, snowflake, kafka, s3, azure_blob, gcs, ftp, ftps, local.

Note. The connector picker offers more combinations than the platform enforces. Where the picker and this page disagree, this page describes what the code does. The gaps are named explicitly in Limitations by engine.

Capability matrix#

Engine	CDC source	CDC mirror destination	ETL source	ETL destination	Export source	Export destination	File-ingest source	Stream destination	Schema evolution
`postgres`	Yes	Yes	Yes	Yes	Yes	No	No	Yes	Yes
`mysql`	Yes	Yes	Yes	Yes	Yes	No	No	Yes	Yes
`mariadb`	No	No	No	No	No	No	No	No	Yes
`cockroachdb`	No	No	Yes	Yes	Yes	No	No	No	Yes
`sqlserver`	Yes	No	No	No	No	No	No	No	No
`singlestore`	No	No	Yes	Yes	Yes	No	No	No	Yes
`snowflake`	No	No	No	No	No	No	No	No	No
`kafka`	No	No	No	No	No	No	No	Yes	No
`s3`	No	No	No	No	No	Yes	Yes	Yes	No
`azure_blob`	No	No	No	No	No	Yes	No	Yes	No
`gcs`	No	No	No	No	No	Yes	No	Yes	No
`ftp`	No	No	No	No	No	Yes	Yes	Yes	No
`ftps`	No	No	No	No	No	Yes	Yes	Yes	No
`local`	No	No	No	No	No	Yes	No	Yes	No

Reading the columns:

CDC source — the engine can be the source of a change-capture table. Only three engines qualify, and only with the strategies listed in CDC strategies: postgres (timestamp, log_pg), mysql (timestamp, log_mysql), sqlserver (log_mssql).
CDC mirror destination — the engine can hold the mirrored table a CDC sync writes into. Only postgres and mysql.
ETL source / destination — the engine has an ETL adapter. Two families exist: the PostgreSQL family (postgres, cockroachdb) and the MySQL family (mysql, singlestore). sqlserver, snowflake and mariadb have no ETL adapter at all and cannot appear on either side of an ETL task.
Export source — the engine can be queried by a data export. postgres, cockroachdb, mysql, singlestore only.
Export destination — where an export writes its files: local, s3, azure_blob, gcs, ftp, ftps.
File-ingest source — where a file ingestion polls for new files: s3, ftp, ftps only.
Stream destination — a fan-out sink. See Stream destinations.
Schema evolution — the engine has an adapter that can issue the additive DDL the Evolve policy needs: postgres, cockroachdb, mysql, mariadb, singlestore. See Schema drift policies.

Warning. mariadb is offered in the export-source picker but always fails at run time with source engine 'mariadb' is not a supported export source yet. Do not build an export on a MariaDB source.

Connection fields, per engine#

Every connector shares one form — host, port, database, username, password. Non-relational engines repurpose those fields. The table below is the authoritative mapping.

Engine	`host`	`port`	`database`	`username`	`password`
`postgres`, `cockroachdb`, `mysql`, `mariadb`, `singlestore`, `sqlserver`, `snowflake`	Server hostname	Server port	Database name	Login	Password
`s3`	AWS region or a full endpoint hostname	`443` (TLS) or `9000` (typical MinIO)	Bucket name	Access key ID	Secret access key
`azure_blob`	Storage account name, or the full account URL	Unused	Container name	Unused	Storage account key or SAS token
`gcs`	Optional custom endpoint; leave blank for Google	Unused	Bucket name	Unused	The entire service-account JSON document
`ftp`, `ftps`	FTP server hostname	`21`	Base directory for all paths	FTP user	FTP password
`kafka`	Comma-separated bootstrap servers (`host:port,host:port`)	Unused	SASL mechanism	SASL username	SASL password
`local`	Unused	Unused	Subdirectory under the server's export root	Unused	Unused

Notes on the object stores:

S3-compatible services (MinIO, Cloudflare R2, Ceph) are configured by putting the service hostname in host and using the Secure toggle to pick https or http. There is no separate "endpoint" field.
azure_blob accepts either a bare account name or a full account URL in host; the credential in password may be a storage key or a SAS token.
gcs takes the whole service-account JSON as the secret. Paste the file contents, not a path to it.

s3 host examples
  eu-central-1                  # AWS region
  minio.example.com             # MinIO, port 9000, Secure off for plain HTTP
  192.0.2.10                    # self-hosted S3-compatible store

Limitations by engine#

mariadb has no change data capture. No strategy accepts it as a source. It is usable only where a schema-evolution adapter is enough.
snowflake has no CDC and exposes no warehouse, role or schema configuration on the connector — only host, database and credentials. There is no ETL adapter for it either.
kafka is sink-only. It cannot be discovered, previewed, or read. It takes no part in schema evolution and ignores truncate instructions.
Object stores are full-load only. There is no incremental read and no Parquet introspection: the platform does not read Parquet footers to infer a schema.
SFTP does not exist anywhere in the platform. The only file-transfer engines are ftp (plain) and ftps (explicit FTPS over TLS). Implicit FTPS on port 990 is not supported. If a source is reachable only over SSH file transfer, land the files in an object store first.
local is a sink. It is not offered in the connector picker; it exists as an export and stream destination writing under the server's export root.

What a connection test checks#

The Test connection action does different work per engine. Knowing what it does not check matters as much as what it does.

Engine group	What the test reports
`postgres`, `cockroachdb`, `sqlserver`, `singlestore`	Connectivity, server version, and a write-privilege introspection. If the login can write, the result carries the warning `current user has write privileges…`.
`mysql`, `mariadb`	The same connectivity + write-privilege introspection, plus a probe of the binary-log settings used by log-based CDC.
`snowflake`	Connectivity only. No privilege or capability detail is reported.
`s3`, `azure_blob`, `gcs`	Reachability only. No bucket policy, retention, or object-listing detail is reported.
`ftp`, `ftps`	Reachability only. No directory-permission detail is reported.
`kafka`	Reports the broker as writable.
`local`	Fails the test if the configured directory is not writable by the server process.

Note. A write-privilege warning on a source connector is informational — it tells you the credentials are broader than a read-only extract needs. Use a read-only role for extract-only connectors.

Warning. The connection test cannot verify every CDC prerequisite. It probes MySQL binary-log settings, but it does not check PostgreSQL wal_level and it does not check whether SQL Server CDC has been enabled on the database or table. Verify those manually — see CDC strategies.

IMAP mailboxes are not connectors. Mail sources are a separate resource type with their own configuration — see Email feeds.