Skip to main content
Version: 2.1.3

Assert

Sink plugin: Assert [Flink]

Description​

A flink sink plugin which can assert illegal data by user defined rules

tip

Engine Supported and plugin name

  • Spark:Assert
  • Flink: Assert

Options​

nametyperequireddefault value
rulesConfigListyes-
rules.field_namestringyes-
rules.field_typestringno-
rules.field_valueConfigListno-
rules.field_value.rule_typestringno-
rules.field_value.rule_valuedoubleno-

rules [ConfigList]​

Rule definition of user's available data. Each rule represents one field validation.

field_name [string]​

field name(string)

field_type [string]​

field type (string), e.g. string,boolean,byte,short,int,long,float,double,char,void,BigInteger,BigDecimal,Instant

field_value [ConfigList]​

A list value rule define the data value validation

rule_type [string]​

The following rules are supported for now NOT_NULL, // value can't be null MIN, // define the minimum value of data MAX, // define the maximum value of data MIN_LENGTH, // define the minimum string length of a string data MAX_LENGTH // define the maximum string length of a string data

rule_value [double]​

the value related to rule type

Example​

the whole config obey with hocon style


Assert {
rules =
[{
field_name = name
field_type = string
field_value = [
{
rule_type = NOT_NULL
},
{
rule_type = MIN_LENGTH
rule_value = 3
},
{
rule_type = MAX_LENGTH
rule_value = 5
}
]
},{
field_name = age
field_type = int
field_value = [
{
rule_type = NOT_NULL
},
{
rule_type = MIN
rule_value = 10
},
{
rule_type = MAX
rule_value = 20
}
]
}
]

}