Amazon SQS sink connector

Support Those Engines

Spark
Flink
SeaTunnel Zeta

Description

Write data to Amazon SQS

Key Features

Sink Options

Name Type Required Default Description
url String Yes - The Queue URL to read from Amazon SQS.
region String No - The AWS region for the SQS service
format String No json Data format. The default format is json. Optional text format, canal-json and debezium-json.If you use json or text format. The default field separator is “, “. If you customize the delimiter, add the “field_delimiter” option.If you use canal format, please refer to canal-json for details.If you use debezium format, please refer to debezium-json for details.
format_error_handle_way String No fail The processing method of data format error. The default value is fail, and the optional value is (fail, skip). When fail is selected, data format error will block and an exception will be thrown. When skip is selected, data format error will skip this line data.
field_delimiter String No , Customize the field delimiter for data format.

Task Example

  1. source {
  2.   FakeSource {
  3.     schema = {
  4.       fields {
  5.         c_map = "map<string, string>"
  6.         c_array = "array<int>"
  7.         c_string = string
  8.         c_boolean = boolean
  9.         c_tinyint = tinyint
  10.         c_smallint = smallint
  11.         c_int = int
  12.         c_bigint = bigint
  13.         c_float = float
  14.         c_double = double
  15.         c_bytes = bytes
  16.         c_date = date
  17.         c_decimal = "decimal(38, 18)"
  18.         c_timestamp = timestamp
  19.         c_row = {
  20.           c_map = "map<string, string>"
  21.           c_array = "array<int>"
  22.           c_string = string
  23.           c_boolean = boolean
  24.           c_tinyint = tinyint
  25.           c_smallint = smallint
  26.           c_int = int
  27.           c_bigint = bigint
  28.           c_float = float
  29.           c_double = double
  30.           c_bytes = bytes
  31.           c_date = date
  32.           c_decimal = "decimal(38, 18)"
  33.           c_timestamp = timestamp
  34.         }
  35.       }
  36.     }
  37.     result_table_name = "fake"
  38.   }
  39. }
  41. sink {
  42.   AmazonSqs {
  43.     url = "http://127.0.0.1:8000"
  44.     region = "us-east-1"
  45.     queue = "queueName"
  46.     format = text
  47.     field_delimiter = "|"  
  48.   }
  49. }