FilterRowKind

FilterRowKind transform plugin

Description

Filter the data by RowKind

Options

name	type	required	default value
include_kinds	array	yes
exclude_kinds	array	yes

include_kinds [array]

The row kinds to include

exclude_kinds [array]

The row kinds to exclude.

You can only config one of include_kinds and exclude_kinds.

common options [string]

Transform plugin common parameters, please refer to Transform Plugin for details

Examples

The RowKind of the data generate by FakeSource is INSERT, If we use FilterRowKind transform and exclude the INSERT data, we will write zero rows into sink.

env {
  job.mode = "BATCH"
}

source {
  FakeSource {
    result_table_name = "fake"
    row.num = 100
    schema = {
      fields {
        id = "int"
        name = "string"
        age = "int"
      }
    }
  }
}

transform {
  FilterRowKind {
    source_table_name = "fake"
    result_table_name = "fake1"
    exclude_kinds = ["INSERT"]
  }
}

sink {
  Console {
    source_table_name = "fake1"
  }
}