FilterRowKind transform plugin

Description

Filter the data by RowKind

Options

name type required default value
include_kinds array yes
exclude_kinds array yes

include_kinds [array]

The row kinds to include

exclude_kinds [array]

The row kinds to exclude.

You can only config one of include_kinds and exclude_kinds.

common options [string]

Transform plugin common parameters, please refer to Transform Plugin for details

Examples

The RowKink of the data generate by FakeSource is INSERT, If we use FilterRowKink transform and exclude the INSERT data, we will write zero rows into sink.

  2. env {
  3.   job.mode = "BATCH"
  4. }
  6. source {
  7.   FakeSource {
  8.     result_table_name = "fake"
  9.     row.num = 100
  10.     schema = {
  11.       fields {
  12.         id = "int"
  13.         name = "string"
  14.         age = "int"
  15.       }
  16.     }
  17.   }
  18. }
  20. transform {
  21.   FilterRowKind {
  22.     source_table_name = "fake"
  23.     result_table_name = "fake1"
  24.     exclude_kinds = ["INSERT"]
  25.   }
  26. }
  28. sink {
  29.   Console {
  30.     source_table_name = "fake1"
  31.   }
  32. }