AmazonSqs

AmazonSqs source connector

Support Those Engines

Spark
Flink
SeaTunnel Zeta

Key Features

Description

Read data from Amazon SQS.

Source Options

Name Type Required Default Description
url String Yes - The Queue URL to read from Amazon SQS.
region String No - The AWS region for the SQS service
schema Config No - The structure of the data, including field names and field types.
format String No json Data format. The default format is json. Optional text format, canal-json and debezium-json.If you use json or text format. The default field separator is “, “. If you customize the delimiter, add the “field_delimiter” option.If you use canal format, please refer to canal-json for details.If you use debezium format, please refer to debezium-json for details.
format_error_handle_way String No fail The processing method of data format error. The default value is fail, and the optional value is (fail, skip). When fail is selected, data format error will block and an exception will be thrown. When skip is selected, data format error will skip this line data.
field_delimiter String No , Customize the field delimiter for data format.
common-options No - Source plugin common parameters, please refer to Source Common Options for details

Task Example

  1. source {
  2.   AmazonSqs {
  3.     url = "http://127.0.0.1:4566"
  4.     region = "us-east-1"
  5.     format = text
  6.     field_delimiter = "#"
  7.     schema = {
  8.       fields {
  9.         artist = string
  10.         c_map = "map<string, array<int>>"
  11.         c_array = "array<int>"
  12.         c_string = string
  13.         c_boolean = boolean
  14.         c_tinyint = tinyint
  15.         c_smallint = smallint
  16.         c_int = int
  17.         c_bigint = bigint
  18.         c_float = float
  19.         c_double = double
  20.         c_decimal = "decimal(30, 8)"
  21.         c_null = "null"
  22.         c_bytes = bytes
  23.         c_date = date
  24.         c_timestamp = timestamp
  25.       }
  26.     }
  27.   }
  28. }
  30. transform {
  31.     # If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
  32.     # please go to https://seatunnel.apache.org/docs/transform-v2/sql
  33. }
  35. sink {
  36.     Console {}
  37. }

Changelog

next version