Skip to main content
Version: 2.3.8

AmazonDynamoDB

AmazonDynamoDB source connector

Description​

Read data from Amazon DynamoDB.

Key features​

Options​

nametyperequireddefault value
urlstringyes-
regionstringyes-
access_key_idstringyes-
secret_access_keystringyes-
tablestringyes-
schemaconfigyes-
common-optionsyes-
scan_item_limitfalse-
parallel_scan_threadsfalse-

url [string]​

The URL to read to Amazon Dynamodb.

region [string]​

The region of Amazon Dynamodb.

accessKeyId [string]​

The access id of Amazon DynamoDB.

secretAccessKey [string]​

The access secret of Amazon DynamoDB.

table [string]​

The table of Amazon DynamoDB.

schema [Config]​

fields [config]​

Amazon Dynamodb is a NOSQL database service of support keys-value storage and document data structure,there is no way to get the data type.Therefore, we must configure schema.

such as:

schema {
fields {
id = int
key_aa = string
key_bb = string
}
}

common options​

Source Plugin common parameters, refer to Source Plugin for details

scan_item_limit​

number of item each scan request should return

parallel_scan_threads​

number of logical segments for parallel scan

Example​

Amazondynamodb {
url = "http://127.0.0.1:8000"
region = "us-east-1"
accessKeyId = "dummy-key"
secretAccessKey = "dummy-secret"
table = "TableName"
schema = {
fields {
artist = string
c_map = "map<string, array<int>>"
c_array = "array<int>"
c_string = string
c_boolean = boolean
c_tinyint = tinyint
c_smallint = smallint
c_int = int
c_bigint = bigint
c_float = float
c_double = double
c_decimal = "decimal(30, 8)"
c_null = "null"
c_bytes = bytes
c_date = date
c_timestamp = timestamp
}
}
}

Changelog​

next version​

  • Add Amazon DynamoDB Source Connector
  • Add source split to Amazondynamodb Connectors