Skip to main content
Version: 2.2.0-beta

MongoDb

MongoDb source connector

Description​

Read data from MongoDB.

tip

Engine Supported and plugin name

  • Spark: MongoDb
  • Flink

Options​

nametyperequireddefault value
readconfig.uristringyes-
readconfig.databasestringyes-
readconfig.collectionstringyes-
readconfig.*stringno-
schemastringno-
common-optionsstringyes-

readconfig.uri [string]​

MongoDB uri

readconfig.database [string]​

MongoDB database

readconfig.collection [string]​

MongoDB collection

readconfig.* [string]​

More other parameters can be configured here, see MongoDB Configuration for details, see the Input Configuration section. The way to specify parameters is to prefix the original parameter name readconfig. For example, the way to set spark.mongodb.input.partitioner is readconfig.spark.mongodb.input.partitioner="MongoPaginateBySizePartitioner" . If you do not specify these optional parameters, the default values of the official MongoDB documentation will be used.

schema [string]​

Because MongoDB does not have the concept of schema, when spark reads MongoDB , it will sample MongoDB data and infer the schema . In fact, this process will be slow and may be inaccurate. This parameter can be manually specified. Avoid these problems. schema is a json string, such as {\"name\":\"string\",\"age\":\"integer\",\"addrs\":{\"country\":\"string\ ",\"city\":\"string\"}}

common options [string]​

Source Plugin common parameters, refer to Source Plugin for details

Example​

mongodb {
readconfig.uri = "mongodb://username:password@127.0.0.1:27017/mypost"
readconfig.database = "mydatabase"
readconfig.collection = "mycollection"
readconfig.spark.mongodb.input.partitioner = "MongoPaginateBySizePartitioner"
schema="{\"name\":\"string\",\"age\":\"integer\",\"addrs\":{\"country\":\"string\",\"city\":\"string\"}}"
result_table_name = "mongodb_result_table"
}