Sql
Sql transform plugin
Description
Use SQL to process data and support engine's UDF function.
This transform both supported by engine Spark and Flink.
Options
name | type | required | default value |
---|---|---|---|
sql | string | yes | - |
common-options | string | no | - |
sql [string]
SQL statement, the table name used in SQL configured in the Source
or Transform
plugin
common options [string]
Transform plugin common parameters, please refer to Transform Plugin for details
Examples
Simple Select
Use the SQL plugin for field deletion. Only the username
and address
fields are reserved, and the remaining fields will be discarded. user_info
is the result_table_name
configured by the previous plugin
sql {
sql = "select username, address from user_info",
}
Use UDF
Use SQL plugin for data processing, use substring
functions to intercept the telephone
field
sql {
sql = "select substring(telephone, 0, 10) from user_info",
}
Use UDAF
Use SQL plugin for data aggregation, use avg functions to perform aggregation operations on the original data set, and take out the average value of the age
field
sql {
sql = "select avg(age) from user_info",
}