Skip to main content
Version: 2.3.1

Intro to schema feature

Why we need schema​

Some NoSQL databases or message queue are not strongly limited schema, so the schema cannot be obtained through the api. At this time, a schema needs to be defined to convert to SeaTunnelRowType and obtain data.

What type supported at now​

Data typeDescription
stringstring
booleanboolean
tinyint-128 to 127 regular. 0 to 255 unsigned*. Specify the maximum number of digits in parentheses.
smallint-32768 to 32767 General. 0 to 65535 unsigned*. Specify the maximum number of digits in parentheses.
intAll numbers from -2,147,483,648 to 2,147,483,647 are allowed.
bigintAll numbers between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 are allowed.
floatFloat-precision numeric data from -1.79E+308 to 1.79E+308.
doubleDouble precision floating point. Handle most decimals.
decimalDOUBLE type stored as a string, allowing a fixed decimal point.
nullnull
bytesbytes.
dateOnly the date is stored. From January 1, 0001 to December 31, 9999.
timeOnly store time. Accuracy is 100 nanoseconds.
timestampStores a unique number that is updated whenever a row is created or modified. timestamp is based on the internal clock and does not correspond to real time. There can only be one timestamp variable per table.
rowRow type,can be nested.
mapA Map is an object that maps keys to values. The key type includes int string boolean tinyint smallint bigint float double decimal date time timestamp null , and the value type includes int string boolean tinyint smallint bigint float double decimal date time timestamp null array map.
arrayA array is a data type that represents a collection of elements. The element type includes int string boolean tinyint smallint bigint float double array map.

How to use schema​

schema defines the format of the data,it containsfields properties. fields define the field properties,it's a K-V key-value pair, the Key is the field name and the value is field type. Here is an example.

source {
FakeSource {
parallelism = 2
result_table_name = "fake"
row.num = 16
schema = {
fields {
id = bigint
c_map = "map<string, smallint>"
c_array = "array<tinyint>"
c_string = string
c_boolean = boolean
c_tinyint = tinyint
c_smallint = smallint
c_int = int
c_bigint = bigint
c_float = float
c_double = double
c_decimal = "decimal(2, 1)"
c_bytes = bytes
c_date = date
c_timestamp = timestamp
}
}
}
}

When we should use it or not​

If there is a schema configuration project in Options,the connector can then customize the schema. Like Fake Pulsar Http source connector etc.