Common parameters of sink connectors

name type required default value
source_table_name string no -
parallelism int no -

source_table_name [string]

When source_table_name is not specified, the current plug-in processes the data set dataset output by the previous plugin in the configuration file;

When source_table_name is specified, the current plug-in is processing the data set corresponding to this parameter.

parallelism [int]

When parallelism is not specified, the parallelism in env is used by default.

When parallelism is specified, it will override the parallelism in env.

Examples

  1. source {
  2.     FakeSourceStream {
  3.       parallelism = 2
  4.       result_table_name = "fake"
  5.       field_name = "name,age"
  6.     }
  7. }
  9. transform {
  10.     Filter {
  11.       source_table_name = "fake"
  12.       fields = [name]
  13.       result_table_name = "fake_name"
  14.     }
  15.     Filter {
  16.       source_table_name = "fake"
  17.       fields = [age]
  18.       result_table_name = "fake_age"
  19.     }
  20. }
  22. sink {
  23.     Console {
  24.       source_table_name = "fake_name"
  25.     }
  26.     Console {
  27.       source_table_name = "fake_age"
  28.     }
  29. }

If the job only have one source and one(or zero) transform and one sink, You do not need to specify source_table_name and result_table_name for connector. If the number of any operator in source, transform and sink is greater than 1, you must specify the source_table_name and result_table_name for each connector in the job.