Introduction

The SeaTunnel provides a powerful speed control feature that allows you to manage the rate at which data is synchronized. This functionality is essential when you need to ensure efficient and controlled data transfer between systems. The speed control is primarily governed by two key parameters: read_limit.rows_per_second and read_limit.bytes_per_second. This document will guide you through the usage of these parameters and how to leverage them effectively.

Support Those Engines

SeaTunnel Zeta
Flink
Spark

Configuration

To use the speed control feature, you need to configure the read_limit.rows_per_second or read_limit.bytes_per_second parameters in your job config.

Example env config in your config file:

  1. env {
  2.     job.mode=STREAMING
  3.     job.name=SeaTunnel_Job
  4.     read_limit.bytes_per_second=7000000
  5.     read_limit.rows_per_second=400
  6. }
  7. source {
  8.     MySQL-CDC {
  9.       // ignore...
  10.     }
  11. }
  12. transform {
  13. }
  14. sink {
  15.     Console {
  16.     }
  17. }

We have placed read_limit.bytes_per_second and read_limit.rows_per_second in the env parameters, completing the speed control configuration. You can configure both of these parameters simultaneously or choose to configure only one of them. The value of each value represents the maximum rate at which each thread is restricted. Therefore, when configuring the respective values, please take into account the parallelism of your tasks.