Introduction

The SeaTunnel provides a rich event listening feature that allows you to manage the status at which data is synchronized. This functionality is crucial when you need to listen job running status(org.apache.seatunnel.api.event). This document will guide you through the usage of these parameters and how to leverage them effectively.

Support Those Engines

SeaTunnel Zeta
Flink
Spark

API

The event API is defined in the org.apache.seatunnel.api.event package.

Event Data API

  • org.apache.seatunnel.api.event.Event - The interface for event data.
  • org.apache.seatunnel.api.event.EventType - The enum for event type.

Event Listener API

You can customize event handler, such as sending events to external systems

  • org.apache.seatunnel.api.event.EventHandler - The interface for event handler, SPI will automatically load subclass from the classpath.

Event Collect API

  • org.apache.seatunnel.api.source.SourceSplitEnumerator - Attached event listener API to report events from SourceSplitEnumerator.
  1. package org.apache.seatunnel.api.source;
  3. public interface SourceSplitEnumerator {
  5.     interface Context {
  7.         /**
  8.          * Get the {@link org.apache.seatunnel.api.event.EventListener} of this enumerator.
  9.          *
  10.          * @return
  11.          */
  12.         EventListener getEventListener();
  13.     }
  14. }
  • org.apache.seatunnel.api.source.SourceReader - Attached event listener API to report events from SourceReader.
  1. package org.apache.seatunnel.api.source;
  3. public interface SourceReader {
  5.     interface Context {
  7.         /**
  8.          * Get the {@link org.apache.seatunnel.api.event.EventListener} of this reader.
  9.          *
  10.          * @return
  11.          */
  12.         EventListener getEventListener();
  13.     }
  14. }
  • org.apache.seatunnel.api.sink.SinkWriter - Attached event listener API to report events from SinkWriter.
  1. package org.apache.seatunnel.api.sink;
  3. public interface SinkWriter {
  5.     interface Context {
  7.         /**
  8.          * Get the {@link org.apache.seatunnel.api.event.EventListener} of this writer.
  9.          *
  10.          * @return
  11.          */
  12.         EventListener getEventListener();
  13.     }
  14. }

Configuration Listener

To use the event listening feature, you need to configure engine config.

Zeta Engine

Example config in your config file(seatunnel.yaml):

  1. seatunnel:
  2.   engine:
  3.     event-report-http:
  4.       url: "http://example.com:1024/event/report"
  5.       headers:
  6.         Content-Type: application/json

You can define the implementation class of org.apache.seatunnel.api.event.EventHandler interface and add to the classpath to automatically load it through SPI.

Support flink version: 1.14.0+

Example: org.apache.seatunnel.api.event.LoggingEventHandler

Spark Engine

You can define the implementation class of org.apache.seatunnel.api.event.EventHandler interface and add to the classpath to automatically load it through SPI.