AmazonDocumentDB

Amazon DocumentDB source connector

Support Connector Version

• Amazon DocumentDB clusters compatible with the MongoDB 4.0 and 5.0 APIs

Support Those Engines

Spark
Flink
SeaTunnel Zeta

Key Features

• batch
• stream
• exactly-once
• column projection
• parallelism
• support user-defined split

Description

Read documents from an existing Amazon DocumentDB collection with a bounded batch scan.

V1 scope for this connector:

• source only
• one database and one collection per source
• explicit schema required; schema inference is out of scope
• optional BSON filter and projection
• one source split; sampling and range-based parallel splitting are out of scope
• sink, CDC/change streams, catalog discovery, and collection creation are out of scope

Supported DataSource Info

The connector can be installed with install-plugin.sh or downloaded from Maven Central.

Datasource	Supported Versions	Dependency
Amazon DocumentDB	MongoDB 4.0 and 5.0 compatibility	Download

Database Dependency

Install the connector plugin before running jobs:

sh bin/install-plugin.sh ${version}

Ensure connector-amazondocumentdb is included in the plugin installation. The connector uses the MongoDB Java synchronous driver 4.7.1. Amazon DocumentDB is normally reachable only from the VPC that contains the cluster, or through network connectivity to that VPC.

Data Type Mapping

You must define the SeaTunnel schema explicitly. BSON values are converted according to the configured field types:

Amazon DocumentDB BSON type	SeaTunnel Data type
Boolean	BOOLEAN
Int32 / Int64 / Double	TINYINT / SMALLINT / INT / BIGINT / FLOAT / DOUBLE
Decimal128	DECIMAL
String / ObjectId / Document	STRING
Date / Timestamp	DATE / TIME / TIMESTAMP
Binary	BYTES
Array	ARRAY
Document	MAP / ROW
Null / Undefined / Decimal128 NaN	null

Decimal128 values are rounded to the configured scale. If the resulting precision exceeds the declared DECIMAL precision, conversion fails with an error instead of silently producing null.

Source Options

Name	Type	Required	Default	Description
uri	string	yes	-	MongoDB-compatible connection URI containing the Amazon DocumentDB endpoint and credentials
database	string	yes	-	Database name
collection	string	yes	-	Collection name
schema	config	yes	-	Explicit data schema
tls	boolean	no	true	Enable TLS for the connection
tls_ca_file	string	yes when tls=true	-	Local path to the PEM CA bundle used to verify the cluster certificate
match.query	string	no	{}	BSON/JSON filter document
match.projection	string	no	-	BSON/JSON projection document
fetch.size	int	no	2048	Number of documents requested from the server per batch; must be greater than zero
common-options	-	no	-	Source plugin common parameters, see Source Common Options

uri [string]

MongoDB-compatible connection URI. Include the username, password, cluster endpoint, and port. The connector rejects an explicit retryWrites=true because Amazon DocumentDB does not support retryable writes, and always applies retryWrites=false after parsing the URI.

Example: mongodb://reader:<password>@sample.cluster-abcdefghijkl.us-east-1.docdb.amazonaws.com:27017/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

database [string]

Name of the existing Amazon DocumentDB database to read.

collection [string]

Name of the existing collection to read.

schema [config]

Explicit SeaTunnel schema. Field names are looked up in each BSON document. Missing or BSON null fields become null.

schema = {
  fields {
    _id = string
    status = string
    amount = "decimal(18,2)"
    created_at = timestamp
    labels = "map<string,string>"
  }
}

tls [boolean]

Enables TLS. The default is true. The connector applies this option after parsing uri, so this option determines the final driver TLS setting.

tls_ca_file [string]

Path to a readable PEM CA bundle. It is required when tls=true. The connector creates a connector-local SSLContext; it does not modify the JVM-wide javax.net.ssl.trustStore system property.

Download the current Amazon trust store from the AWS documentation.

match.query [string]

BSON/JSON filter passed to find, for example {"status": "OPEN"}. The default {} reads all documents.

match.projection [string]

BSON/JSON projection passed to find, for example {"_id": 1, "status": 1, "amount": 1}.

fetch.size [int]

Driver batch-size hint for the server cursor. A larger value can reduce round trips but increases the amount of data buffered by the driver.

common-options

Source plugin common parameters, refer to Source Common Options.

Tips

1. Use a read-only Amazon DocumentDB user and keep credentials out of checked-in job files.
   
2. TLS is enabled by default. Use the current AWS CA bundle and rotate the local file when AWS updates its trust chain.
   
3. V1 creates one split. Increasing source parallelism does not parallelize the collection scan.
   
4. Split state contains the filter and projection, not cursor progress. Recovery restarts the full collection scan from the beginning—even if the failed attempt was almost complete—so downstream writes must be idempotent or use a truncate-and-reload strategy to avoid duplicates.
   
5. Push selective predicates into match.query and include every schema field you need in match.projection.

How to Create an Amazon DocumentDB Data Synchronization Job

The following batch job reads an existing collection and prints rows to the local client:

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  AmazonDocumentDB {
    uri = "mongodb://reader:<password>@sample.cluster-abcdefghijkl.us-east-1.docdb.amazonaws.com:27017/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false"
    database = "app-db"
    collection = "orders"
    tls = true
    tls_ca_file = "/opt/seatunnel/certs/global-bundle.pem"
    fetch.size = 2048
    schema = {
      fields {
        _id = string
        status = string
        amount = "decimal(18,2)"
        created_at = timestamp
      }
    }
  }
}

sink {
  Console {}
}

Filter and project documents

source {
  AmazonDocumentDB {
    uri = "mongodb://reader:<password>@sample.cluster-abcdefghijkl.us-east-1.docdb.amazonaws.com:27017/?replicaSet=rs0&retryWrites=false"
    database = "app-db"
    collection = "orders"
    tls_ca_file = "/opt/seatunnel/certs/global-bundle.pem"
    match.query = '{"status": "OPEN", "amount": {"$gt": 100}}'
    match.projection = '{"_id": 1, "status": 1, "amount": 1}'
    fetch.size = 512
    schema = {
      fields {
        _id = string
        status = string
        amount = "decimal(18,2)"
      }
    }
  }
}

Changelog

Change Log

Change	Commit	Version
[Feature][Connector-V2] Add Amazon DocumentDB source connector	-	Next