Skip to main content
Version: Next

Milvus

Milvus source connector

Description

This Milvus source connector reads data from Milvus or Zilliz Cloud, it has the following features:

  • support read and write data by partition
  • support read dynamic schema data into Metadata Column
  • json data will be converted to json string and sink as json as well
  • retry automatically to bypass ratelimit and grpc limit

Key Features

Data Type Mapping

Milvus Data TypeSeaTunnel Data Type
INT8TINYINT
INT16SMALLINT
INT32INT
INT64BIGINT
FLOATFLOAT
DOUBLEDOUBLE
BOOLBOOLEAN
JSONSTRING
ARRAYARRAY
VARCHARSTRING
FLOAT_VECTORFLOAT_VECTOR
BINARY_VECTORBINARY_VECTOR
FLOAT16_VECTORFLOAT16_VECTOR
BFLOAT16_VECTORBFLOAT16_VECTOR
SPARSE_FLOAT_VECTORSPARSE_FLOAT_VECTOR

Source Options

NameTypeRequiredDefaultDescription
urlStringYes-The URL to connect to Milvus or Zilliz Cloud.
tokenStringYes-User:password
databaseStringYesdefaultRead data from which database.
collectionStringNo-If set, will only read one collection, otherwise will read all collections under database.

Task Example

source {
Milvus {
url = "http://127.0.0.1:19530"
token = "username:password"
database = "default"
}
}

Changelog