Skip to main content
Version: 2.1.2

Sql

Description​

Use SQL to process data and support engine's UDF function.

tip

This transform both supported by engine Spark and Flink.

Options​

nametyperequireddefault value
sqlstringyes-
common-optionsstringno-

sql [string]​

SQL statement, the table name used in SQL configured in the Source or Transform plugin

common options [string]​

Transform plugin common parameters, please refer to Transform Plugin for details

Examples​

Simple Select​

Use the SQL plugin for field deletion. Only the username and address fields are reserved, and the remaining fields will be discarded. user_info is the result_table_name configured by the previous plugin

sql {
    sql = "select username, address from user_info",
}

Use UDF​

Use SQL plugin for data processing, use substring functions to intercept the telephone field

sql {
    sql = "select substring(telephone, 0, 10) from user_info",
}

Use UDAF​

Use SQL plugin for data aggregation, use avg functions to perform aggregation operations on the original data set, and take out the average value of the age field

sql {
    sql = "select avg(age) from user_info",
    table_name = "user_info"
}