跳到主要内容
版本:Next

Milvus

Milvus sink connector

Description

This Milvus sink connector write data to Milvus or Zilliz Cloud, it has the following features:

  • support read and write data by partition
  • support write dynamic schema data from Metadata Column
  • json data will be converted to json string and sink as json as well
  • retry automatically to bypass ratelimit and grpc limit

Key Features

Data Type Mapping

Milvus Data TypeSeaTunnel Data Type
INT8TINYINT
INT16SMALLINT
INT32INT
INT64BIGINT
FLOATFLOAT
DOUBLEDOUBLE
BOOLBOOLEAN
JSONSTRING
ARRAYARRAY
VARCHARSTRING
FLOAT_VECTORFLOAT_VECTOR
BINARY_VECTORBINARY_VECTOR
FLOAT16_VECTORFLOAT16_VECTOR
BFLOAT16_VECTORBFLOAT16_VECTOR
SPARSE_FLOAT_VECTORSPARSE_FLOAT_VECTOR

Sink Options

NameTypeRequiredDefaultDescription
urlStringYes-The URL to connect to Milvus or Zilliz Cloud.
tokenStringYes-User:password
databaseStringNo-Write data to which database, default is source database.
schema_save_modeenumNoCREATE_SCHEMA_WHEN_NOT_EXISTAuto create table when table not exist.
enable_auto_idbooleanNofalsePrimary key column enable autoId.
enable_upsertbooleanNofalseUpsert data not insert.
enable_dynamic_fieldbooleanNotrueEnable create table with dynamic field.
batch_sizeintNo1000Write batch size.
partition_keyStringNoMilvus partition key field

Task Example

sink {
Milvus {
url = "http://127.0.0.1:19530"
token = "username:password"
batch_size = 1000
}
}