InfluxDB
InfluxDB sink connector
Description
Write data to InfluxDB.
Key features
Options
name | type | required | default value |
---|---|---|---|
url | string | yes | - |
database | string | yes | |
measurement | string | yes | |
username | string | no | - |
password | string | no | - |
key_time | string | no | processing time |
key_tags | array | no | exclude field & key_time |
batch_size | int | no | 1024 |
max_retries | int | no | - |
retry_backoff_multiplier_ms | int | no | - |
connect_timeout_ms | long | no | 15000 |
common-options | config | no | - |
url
the url to connect to influxDB e.g.
http://influxdb-host:8086
database [string]
The name of influxDB
database
measurement [string]
The name of influxDB
measurement
username [string]
influxDB
user username
password [string]
influxDB
user password
key_time [string]
Specify field-name of the influxDB
measurement timestamp in SeaTunnelRow. If not specified, use processing-time as timestamp
key_tags [array]
Specify field-name of the influxDB
measurement tags in SeaTunnelRow.
If not specified, include all fields with influxDB
measurement field
batch_size [int]
For batch writing, when the number of buffers reaches the number of batch_size
or the time reaches checkpoint.interval
, the data will be flushed into the influxDB
max_retries [int]
The number of retries to flush failed
retry_backoff_multiplier_ms [int]
Using as a multiplier for generating the next delay for backoff
max_retry_backoff_ms [int]
The amount of time to wait before attempting to retry a request to influxDB
connect_timeout_ms [long]
the timeout for connecting to InfluxDB, in milliseconds
common options
Sink plugin common parameters, please refer to Sink Common Options for details
Examples
sink {
InfluxDB {
url = "http://influxdb-host:8086"
database = "test"
measurement = "sink"
key_time = "time"
key_tags = ["label"]
batch_size = 1
}
}
Multiple table
example1
env {
parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
Mysql-CDC {
base-url = "jdbc:mysql://127.0.0.1:3306/seatunnel"
username = "root"
password = "******"
table-names = ["seatunnel.role","seatunnel.user","galileo.Bucket"]
}
}
transform {
}
sink {
InfluxDB {
url = "http://influxdb-host:8086"
database = "test"
measurement = "${table_name}_test"
}
}
Changelog
Change Log
Change | Commit | Version |
---|---|---|
[Improve] influxdb options (#8966) | https://github.com/apache/seatunnel/commit/9f498b813 | 2.3.10 |
[Improve] restruct connector common options (#8634) | https://github.com/apache/seatunnel/commit/f3499a6ee | 2.3.10 |
[Improve][dist]add shade check rule (#8136) | https://github.com/apache/seatunnel/commit/51ef80001 | 2.3.9 |
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786) | https://github.com/apache/seatunnel/commit/6b7c53d03 | 2.3.9 |
[Improve] Improve some connectors prepare check error message (#7465) | https://github.com/apache/seatunnel/commit/6930a25ed | 2.3.8 |
[Improve][Connector] Add multi-table sink option check (#7360) | https://github.com/apache/seatunnel/commit/2489f6446 | 2.3.7 |
[Feature][Core] Support using upstream table placeholders in sink options and auto replacement (#7131) | https://github.com/apache/seatunnel/commit/c4ca74122 | 2.3.6 |
Support multi-table sink feature for influxdb (#6278) | https://github.com/apache/seatunnel/commit/56f13e920 | 2.3.5 |
[Improve][Zeta] Add classloader cache mode to fix metaspace leak (#6355) | https://github.com/apache/seatunnel/commit/9c3c2f183 | 2.3.5 |
[Test][E2E] Add thread leak check for connector (#5773) | https://github.com/apache/seatunnel/commit/1f2f3fc5f | 2.3.4 |
[BugFix][InfluxDBSource] Resolve invalid SQL in initColumnsIndex method caused by direct QUERY_LIMIT appendage with 'tz' function. (#4829) | https://github.com/apache/seatunnel/commit/deed9c62c | 2.3.4 |
[Improve][Common] Introduce new error define rule (#5793) | https://github.com/apache/seatunnel/commit/9d1b2582b | 2.3.4 |
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755) | https://github.com/apache/seatunnel/commit/8de740810 | 2.3.4 |
Support config column/primaryKey/constraintKey in schema (#5564) | https://github.com/apache/seatunnel/commit/eac76b4e5 | 2.3.4 |
[Improve][Connector-V2] Remove scheduler in InfluxDB sink (#5271) | https://github.com/apache/seatunnel/commit/f459f500c | 2.3.4 |
[Improve][CheckStyle] Remove useless 'SuppressWarnings' annotation of checkstyle. (#5260) | https://github.com/apache/seatunnel/commit/51c0d709b | 2.3.4 |
Merge branch 'dev' into merge/cdc | https://github.com/apache/seatunnel/commit/4324ee191 | 2.3.1 |
[Improve][Project] Code format with spotless plugin. | https://github.com/apache/seatunnel/commit/423b58303 | 2.3.1 |
[improve][api] Refactoring schema parse (#4157) | https://github.com/apache/seatunnel/commit/b2f573a13 | 2.3.1 |
[Improve][build] Give the maven module a human readable name (#4114) | https://github.com/apache/seatunnel/commit/d7cd60105 | 2.3.1 |
[Improve][Project] Code format with spotless plugin. (#4101) | https://github.com/apache/seatunnel/commit/a2ab16656 | 2.3.1 |
[Improve][SourceConnector] Unifie InfluxDB source fields to schema (#3897) | https://github.com/apache/seatunnel/commit/85a984a64 | 2.3.1 |
[Feature][Connector] add get source method to all source connector (#3846) | https://github.com/apache/seatunnel/commit/417178fb8 | 2.3.1 |
[Feature][API & Connector & Doc] add parallelism and column projection interface (#3829) | https://github.com/apache/seatunnel/commit/b9164b8ba | 2.3.1 |
[Hotfix][OptionRule] Fix option rule about all connectors (#3592) | https://github.com/apache/seatunnel/commit/226dc6a11 | 2.3.0 |
[Improve][Connector-V2][Influxdb] Unified exception for influxdb source & sink connector (#3558) | https://github.com/apache/seatunnel/commit/4686f35d6 | 2.3.0 |
[Feature][Connector][influx] Expose configurable options in influx db (#3392) | https://github.com/apache/seatunnel/commit/b247ff0ae | 2.3.0 |
[Feature][Connector-V2] influxdb sink connector (#3174) | https://github.com/apache/seatunnel/commit/630e88479 | 2.3.0 |
[Feature][Connector-V2] Add influxDB connector source (#2697) | https://github.com/apache/seatunnel/commit/1d70ea308 | 2.3.0-beta |