Console sink connector

Support Connector Version

  • All versions

Support Those Engines

Spark
Flink
SeaTunnel Zeta

Description

Used to send data to Console. Both support streaming and batch mode.

For example, if the data from upstream is [age: 12, name: jared], the content send to console is the following: {"name":"jared","age":17}

Key Features

Options

Name Type Required Default Description
common-options No - Sink plugin common parameters, please refer to Sink Common Options for details
log.print.data boolean No - Flag to determine whether data should be printed in the logs. The default value is true
log.print.delay.ms int No - Delay in milliseconds between printing each data item to the logs. The default value is 0.

Task Example

Simple:

This is a randomly generated data, written to the console, with a degree of parallelism of 1

  1. env {
  2.   parallelism = 1
  3.   job.mode = "STREAMING"
  4. }
  6. source {
  7.   FakeSource {
  8.     result_table_name = "fake"
  9.     schema = {
  10.       fields {
  11.         name = "string"
  12.         age = "int"
  13.       }
  14.     }
  15.   }
  16. }
  18. sink {
  19.   Console {
  20.     source_table_name = "fake"
  21.   }
  22. }

Multiple Sources Simple:

This is a multiple source and you can specify a data source to write to the specified end

  1. env {
  2.   parallelism = 1
  3.   job.mode = "STREAMING"
  4. }
  6. source {
  7.   FakeSource {
  8.     result_table_name = "fake1"
  9.     schema = {
  10.       fields {
  11.         id = "int"
  12.         name = "string"
  13.         age = "int"
  14.         sex = "string"
  15.       }
  16.     }
  17.   }
  18.    FakeSource {
  19.     result_table_name = "fake2"
  20.     schema = {
  21.       fields {
  22.         name = "string"
  23.         age = "int"
  24.       }
  25.     }
  26.   }
  27. }
  29. sink {
  30.   Console {
  31.     source_table_name = "fake1"
  32.   }
  33.   Console {
  34.     source_table_name = "fake2"
  35.   }
  36. }

Console Sample Data

This is a printout from our console

  1. 2022-12-19 11:01:45,417 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - output rowType: name<STRING>, age<INT>
  2. 2022-12-19 11:01:46,489 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=1: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: CpiOd, 8520946
  3. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=2: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: eQqTs, 1256802974
  4. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=3: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: UsRgO, 2053193072
  5. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=4: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: jDQJj, 1993016602
  6. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=5: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: rqdKp, 1392682764
  7. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=6: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: wCoWN, 986999925
  8. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=7: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: qomTU, 72775247
  9. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=8: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: jcqXR, 1074529204
  10. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=9: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: AkWIO, 1961723427
  11. 2022-12-19 11:01:46,490 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0 rowIndex=10: SeaTunnelRow#tableId=-1 SeaTunnelRow#kind=INSERT: hBoib, 929089763