Skip to main content
Version: 2.3.7

Filter

Filter transform plugin

Description

Filter the field.

Options

nametyperequireddefault value
include_fieldsarrayno
exclude_fieldsarrayno

Notice, you must set one and only one of include_fields and exclude_fields properties

include_fields [array]

The list of fields that need to be kept. Fields not in the list will be deleted.

exclude_fields [array]

The list of fields that need to be deleted. Fields not in the list will be kept.

common options [string]

Transform plugin common parameters, please refer to Transform Plugin for details

Example

The data read from source is a table like this:

nameagecard
Joy Ding20123
May Ding20123
Kin Dom20123
Joy Dom20123

we want to keep the field named name, card, we can add a Filter Transform like below:

transform {
Filter {
source_table_name = "fake"
result_table_name = "fake1"
include_fields = [name, card]
}
}

Or we can delete the field named age by adding a Filter Transform with exclude_fields field set like below:

transform {
Filter {
source_table_name = "fake"
result_table_name = "fake1"
exclude_fields = [age]
}
}

It is useful when you want to delete a small number of fields from a large table with tons of fields.

Then the data in result table fake1 will like this

namecard
Joy Ding123
May Ding123
Kin Dom123
Joy Dom123

Changelog

new version

  • Add Filter Transform Connector