Milvus
Milvus source connector
Descriptionâ
This Milvus source connector reads data from Milvus or Zilliz Cloud, it has the following features:
- support read and write data by partition
- support read dynamic schema data into Metadata Column
- json data will be converted to json string and sink as json as well
- retry automatically to bypass ratelimit and grpc limit
Key Featuresâ
Data Type Mappingâ
Milvus Data Type | SeaTunnel Data Type |
---|---|
INT8 | TINYINT |
INT16 | SMALLINT |
INT32 | INT |
INT64 | BIGINT |
FLOAT | FLOAT |
DOUBLE | DOUBLE |
BOOL | BOOLEAN |
JSON | STRING |
ARRAY | ARRAY |
VARCHAR | STRING |
FLOAT_VECTOR | FLOAT_VECTOR |
BINARY_VECTOR | BINARY_VECTOR |
FLOAT16_VECTOR | FLOAT16_VECTOR |
BFLOAT16_VECTOR | BFLOAT16_VECTOR |
SPARSE_FLOAT_VECTOR | SPARSE_FLOAT_VECTOR |
Source Optionsâ
Name | Type | Required | Default | Description |
---|---|---|---|---|
url | String | Yes | - | The URL to connect to Milvus or Zilliz Cloud. |
token | String | Yes | - | User:password |
database | String | Yes | default | Read data from which database. |
collection | String | No | - | If set, will only read one collection, otherwise will read all collections under database. |
Task Exampleâ
source {
Milvus {
url = "http://127.0.0.1:19530"
token = "username:password"
database = "default"
}
}