跳到主要内容
版本:2.3.0-beta

Http

Http source connector

Description

Used to read data from Http.

Key features

Options

nametyperequireddefault value
urlStringYes-
schemaConfigNo-
schema.fieldsConfigNo-
formatStringNojson
methodStringNoget
headersMapNo-
paramsMapNo-
bodyStringNo-
poll_interval_msintNo-
retryintNo-
retry_backoff_multiplier_msintNo100
retry_backoff_max_msintNo10000
common-optionsNo-

url [String]

http request url

method [String]

http request method, only supports GET, POST method.

headers [Map]

http headers

params [Map]

http params

body [String]

http body

poll_interval_ms [int]

request http api interval(millis) in stream mode

retry [int]

The max retry times if request http return to IOException

retry_backoff_multiplier_ms [int]

The retry-backoff times(millis) multiplier if request http failed

retry_backoff_max_ms [int]

The maximum retry-backoff times(millis) if request http failed

format [String]

the format of upstream data, now only support json text, default json.

when you assign format is json, you should also assign schema option, for example:

upstream data is the following:


{"code": 200, "data": "get success", "success": true}

you should assign schema as the following:


schema {
fields {
code = int
data = string
success = boolean
}
}

connector will generate data as the following:

codedatasuccess
200get successtrue

when you assign format is text, connector will do nothing for upstream data, for example:

upstream data is the following:


{"code": 200, "data": "get success", "success": true}

connector will generate data as the following:

content
{"code": 200, "data": "get success", "success": true}

schema [Config]

fields [Config]

the schema fields of upstream data

common options

Source plugin common parameters, please refer to Source Common Options for details

Example

simple:

Http {
url = "https://tyrantlucifer.com/api/getDemoData"
schema {
fields {
code = int
message = string
data = string
ok = boolean
}
}
}

Changelog

2.2.0-beta 2022-09-26

  • Add Http Source Connector