Http
Http source connector
Description
Used to read data from Http.
Key features
Options
name | type | required | default value |
---|---|---|---|
url | String | Yes | - |
schema | Config | No | - |
schema.fields | Config | No | - |
format | String | No | json |
method | String | No | get |
headers | Map | No | - |
params | Map | No | - |
body | String | No | - |
poll_interval_ms | int | No | - |
retry | int | No | - |
retry_backoff_multiplier_ms | int | No | 100 |
retry_backoff_max_ms | int | No | 10000 |
common-options | No | - |
url [String]
http request url
method [String]
http request method, only supports GET, POST method.
headers [Map]
http headers
params [Map]
http params
body [String]
http body
poll_interval_ms [int]
request http api interval(millis) in stream mode
retry [int]
The max retry times if request http return to IOException
retry_backoff_multiplier_ms [int]
The retry-backoff times(millis) multiplier if request http failed
retry_backoff_max_ms [int]
The maximum retry-backoff times(millis) if request http failed
format [String]
the format of upstream data, now only support json
text
, default json
.
when you assign format is json
, you should also assign schema option, for example:
upstream data is the following:
{"code": 200, "data": "get success", "success": true}
you should assign schema as the following:
schema {
fields {
code = int
data = string
success = boolean
}
}
connector will generate data as the following:
code | data | success |
---|---|---|
200 | get success | true |
when you assign format is text
, connector will do nothing for upstream data, for example:
upstream data is the following:
{"code": 200, "data": "get success", "success": true}
connector will generate data as the following:
content |
---|
{"code": 200, "data": "get success", "success": true} |
schema [Config]
fields [Config]
the schema fields of upstream data
common options
Source plugin common parameters, please refer to Source Common Options for details
Example
simple:
Http {
url = "https://tyrantlucifer.com/api/getDemoData"
schema {
fields {
code = int
message = string
data = string
ok = boolean
}
}
}
Changelog
2.2.0-beta 2022-09-26
- Add Http Source Connector