Elasticsearch

Output plugin : Elasticsearch

• Author: InterestingLab
• Homepage: https://interestinglab.github.io/seatunnel-docs
• Version: 1.0.0

Description

Write Rows to Elasticsearch. Support Elasticsearch >= 2.X

Options

name	type	required	default value
hosts	array	yes	-
index_type	string	no	_doc
index_time_format	string	no	yyyy.MM.dd
index	string	no	seatunnel
es	string	no	-

hosts [array]

Elasticsearch hosts, format as host:port. For example, ["host1:9200", "host2:9200"]

index_type [string]

Elasticsearch index type

index_time_format [string]

Elasticsearch time format. If index likes xxxx-${now}, index_time_format can be used to specify the format of index, default is yyyy.MM.dd. The commonly used time formats are listed below:

Symbol	Description
y	Year
M	Month
d	Day of month
H	Hour in day (0-23)
m	Minute in hour
s	Second in minute

The detailed time format syntax:Java SimpleDateFormat.

index [string]

Elasticsearch index name, if you want to generate index based on time, you need to specify the field like seatunnel-${now}. now means current time.

es.* [string]

You can also specify multiple Elasticsearch's parameters described in [Elasticsearch Configuration](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping.

The way to specify parameters is to use the prefix "es" before the parameter. For example, batch.size.entries is specified as: es.batch.size.entries = 1000.If you do not specify these parameters, it will be set the default values according to Elasticsearch documentation

Examples

elasticsearch {
    hosts = ["localhost:9200"]
    index = "seatunnel"
}

Index name is seatunnel

elasticsearch {
    hosts = ["localhost:9200"]
    index = "seatunnel-${now}"
    es.batch.size.entries = 100000
    index_time_format = "yyyy.MM.dd"
}

Create index by day. For example: seatunnel-2017.11.03