Skip to main content
Version: 2.3.8

Filter

Filter transform plugin

Description​

Filter the field.

Options​

nametyperequireddefault value
include_fieldsarrayno
exclude_fieldsarrayno

Notice, you must set one and only one of include_fields and exclude_fields properties

include_fields [array]​

The list of fields that need to be kept. Fields not in the list will be deleted.

exclude_fields [array]​

The list of fields that need to be deleted. Fields not in the list will be kept.

common options [string]​

Transform plugin common parameters, please refer to Transform Plugin for details

Example​

The data read from source is a table like this:

nameagecard
Joy Ding20123
May Ding20123
Kin Dom20123
Joy Dom20123

we want to keep the field named name, card, we can add a Filter Transform like below:

transform {
Filter {
source_table_name = "fake"
result_table_name = "fake1"
include_fields = [name, card]
}
}

Or we can delete the field named age by adding a Filter Transform with exclude_fields field set like below:

transform {
Filter {
source_table_name = "fake"
result_table_name = "fake1"
exclude_fields = [age]
}
}

It is useful when you want to delete a small number of fields from a large table with tons of fields.

Then the data in result table fake1 will like this

namecard
Joy Ding123
May Ding123
Kin Dom123
Joy Dom123

Changelog​

new version​

  • Add Filter Transform Connector