Skip to main content
Version: 2.3.0

Elasticsearch

Elasticsearch source connector

Description​

Used to read data from Elasticsearch.

support version >= 2.x and < 8.x.

Key features​

Options​

nametyperequireddefault value
hostsarrayyes-
usernamestringno-
passwordstringno-
indexstringyes-
sourcearrayno-
scroll_timestringno1m
scroll_sizeintno100
schemano-

hosts [array]​

Elasticsearch cluster http address, the format is host:port, allowing multiple hosts to be specified. Such as ["host1:9200", "host2:9200"].

username [string]​

x-pack username.

password [string]​

x-pack password.

index [string]​

Elasticsearch index name, support * fuzzy matching.

source [array]​

The fields of index. You can get the document id by specifying the field _id.If sink _id to other index,you need specify an alias for _id due to the Elasticsearch limit. If you don't config source, you must config schema.

scroll_time [String]​

Amount of time Elasticsearch will keep the search context alive for scroll requests.

scroll_size [int]​

Maximum number of hits to be returned with each Elasticsearch scroll request.

schema​

The structure of the data, including field names and field types. If you don't config schema, you must config source.

Examples​

simple

Elasticsearch {
    hosts = ["localhost:9200"]
    index = "seatunnel-*"
    source = ["_id","name","age"]
}

complex

Elasticsearch {
    hosts = ["elasticsearch:9200"]
    index = "st_index"
    schema = {
        fields {
            c_map = "map<string, tinyint>"
            c_array = "array<tinyint>"
            c_string = string
            c_boolean = boolean
            c_tinyint = tinyint
            c_smallint = smallint
            c_int = int
            c_bigint = bigint
            c_float = float
            c_double = double
            c_decimal = "decimal(2, 1)"
            c_bytes = bytes
            c_date = date
            c_timestamp = timestamp
        }
    }
}

Changelog​

2.3.0 2022-12-30​

  • Add Elasticsearch Source Connector