跳到主要内容
版本:2.3.1

Notion

Notion source connector

Description

Used to read data from Notion.

Key features

Options

nametyperequireddefault value
urlStringYes-
passwordStringYes-
versionStringYes-
methodStringNoget
schema.fieldsConfigNo-
formatStringNojson
paramsMapNo-
bodyStringNo-
json_fieldConfigNo-
content_jsonStringNo-
poll_interval_msintNo-
retryintNo-
retry_backoff_multiplier_msintNo100
retry_backoff_max_msintNo10000
common-optionsconfigNo-

url [String]

http request url

password [String]

API key for login, you can get more detail at this link:

https://developers.notion.com/docs/authorization

version [String]

The Notion API is versioned. API versions are named for the date the version is released

method [String]

http request method, only supports GET, POST method

params [Map]

http params

body [String]

http body

poll_interval_ms [int]

request http api interval(millis) in stream mode

retry [int]

The max retry times if request http return to IOException

retry_backoff_multiplier_ms [int]

The retry-backoff times(millis) multiplier if request http failed

retry_backoff_max_ms [int]

The maximum retry-backoff times(millis) if request http failed

format [String]

the format of upstream data, now only support json text, default json.

when you assign format is json, you should also assign schema option, for example:

upstream data is the following:

{
"code": 200,
"data": "get success",
"success": true
}

you should assign schema as the following:


schema {
fields {
code = int
data = string
success = boolean
}
}

connector will generate data as the following:

codedatasuccess
200get successtrue

when you assign format is text, connector will do nothing for upstream data, for example:

upstream data is the following:

{
"code": 200,
"data": "get success",
"success": true
}

connector will generate data as the following:

content
{"code": 200, "data": "get success", "success": true}

schema [Config]

fields [Config]

the schema fields of upstream data

content_json [String]

This parameter can get some json data.If you only need the data in the 'book' section, configure content_field = "$.store.book.*".

If your return data looks something like this.

{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

You can configure content_field = "$.store.book.*" and the result returned looks like this:

[
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
]

Then you can get the desired result with a simpler schema,like

Http {
url = "http://mockserver:1080/contentjson/mock"
method = "GET"
format = "json"
content_field = "$.store.book.*"
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}

Here is an example:

json_field [Config]

This parameter helps you configure the schema,so this parameter must be used with schema.

If your data looks something like this:

{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

You can get the contents of 'book' by configuring the task as follows:

source {
Http {
url = "http://mockserver:1080/jsonpath/mock"
method = "GET"
format = "json"
json_field = {
category = "$.store.book[*].category"
author = "$.store.book[*].author"
title = "$.store.book[*].title"
price = "$.store.book[*].price"
}
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}
}

common options

Source plugin common parameters, please refer to Source Common Options for details

Example

Notion {
url = "https://api.notion.com/v1/users"
password = "Seatunnel-test"
version = "2022-06-28"
content_field = "$.results.*"
schema = {
fields {
object = string
id = string
type = string
person = {
email = string
}
avatar_url = string
}
}
}

Changelog

next version

  • Add Notion Source Connector