AmazonDynamoDB

AmazonDynamoDB source connector

Description

Read data from Amazon DynamoDB.

Key features

• batch
• stream
• exactly-once
• schema projection
• parallelism
• support user-defined split

Options

name	type	required	default value
url	string	yes	-
region	string	yes	-
access_key_id	string	yes	-
secret_access_key	string	yes	-
table	string	yes	-
schema	config	yes	-
common-options		yes	-

url [string]

The URL to read to Amazon Dynamodb.

region [string]

The region of Amazon Dynamodb.

accessKeyId [string]

The access id of Amazon DynamoDB.

secretAccessKey [string]

The access secret of Amazon DynamoDB.

table [string]

The table of Amazon DynamoDB.

schema [Config]

fields [config]

Amazon Dynamodb is a NOSQL database service of support keys-value storage and document data structure,there is no way to get the data type.Therefore, we must configure schema.

such as:

schema {
  fields {
    id = int
    key_aa = string
    key_bb = string
  }
}

common options

Source Plugin common parameters, refer to Source Plugin for details

Example

  Amazondynamodb {
    url = "http://127.0.0.1:8000"
    region = "us-east-1"
    accessKeyId = "dummy-key"
    secretAccessKey = "dummy-secret"
    table = "TableName"
    schema = {
      fields {
        artist = string
        c_map = "map<string, array<int>>"
        c_array = "array<int>"
        c_string = string
        c_boolean = boolean
        c_tinyint = tinyint
        c_smallint = smallint
        c_int = int
        c_bigint = bigint
        c_float = float
        c_double = double
        c_decimal = "decimal(30, 8)"
        c_null = "null"
        c_bytes = bytes
        c_date = date
        c_timestamp = timestamp
      }
    }
  }

Changelog

2.3.0 2022-12-30

• Add Amazon DynamoDB Source Connector