UDF of SQL transform plugin

Description

Use UDF SPI to extends the SQL transform functions lib.

UDF API

  1. package org.apache.seatunnel.transform.sql.zeta;
  3. public interface ZetaUDF {
  4.     /**
  5.      * Function name
  6.      *
  7.      * @return function name
  8.      */
  9.     String functionName();
  11.     /**
  12.      * The type of function result
  13.      *
  14.      * @param argsType input arguments type
  15.      * @return result type
  16.      */
  17.     SeaTunnelDataType<?> resultType(List<SeaTunnelDataType<?>> argsType);
  19.     /**
  20.      * Evaluate
  21.      *
  22.      * @param args input arguments
  23.      * @return result value
  24.      */
  25.     Object evaluate(List<Object> args);
  26. }

UDF Implements Example

Add these dependencies and provided scope to your maven project:

  2. <dependencies>
  3.     <dependency>
  4.         <groupId>org.apache.seatunnel</groupId>
  5.         <artifactId>seatunnel-transforms-v2</artifactId>
  6.         <version>2.3.2</version>
  7.         <scope>provided</scope>
  8.     </dependency>
  9.     <dependency>
  10.         <groupId>org.apache.seatunnel</groupId>
  11.         <artifactId>seatunnel-api</artifactId>
  12.         <version>2.3.2</version>
  13.         <scope>provided</scope>
  14.     </dependency>
  15.     <dependency>
  16.         <groupId>com.google.auto.service</groupId>
  17.         <artifactId>auto-service</artifactId>
  18.         <version>1.0.1</version>
  19.         <scope>provided</scope>
  20.     </dependency>
  21. </dependencies>

Add a Java Class implements of ZetaUDF like this:

  2. @AutoService(ZetaUDF.class)
  3. public class ExampleUDF implements ZetaUDF {
  4.     @Override
  5.     public String functionName() {
  6.         return "EXAMPLE";
  7.     }
  9.     @Override
  10.     public SeaTunnelDataType<?> resultType(List<SeaTunnelDataType<?>> argsType) {
  11.         return BasicType.STRING_TYPE;
  12.     }
  14.     @Override
  15.     public Object evaluate(List<Object> args) {
  16.         String arg = (String) args.get(0);
  17.         if (arg == null) return null;
  18.         return "UDF: " + arg;
  19.     }
  20. }

Package the UDF project and copy the jar to the path: ${SEATUNNEL_HOME}/lib. And if your UDF use third party library, you also need put it to ${SEATUNNEL_HOME}/lib.
If you use cluster mode, you need put the lib to all your node’s ${SEATUNNEL_HOME}/lib folder and re-start the cluster.

Example

The data read from source is a table like this:

id name age
1 Joy Ding 20
2 May Ding 21
3 Kin Dom 24
4 Joy Dom 22

We use UDF of SQL query to transform the source data like this:

  1. transform {
  2.   Sql {
  3.     source_table_name = "fake"
  4.     result_table_name = "fake1"
  5.     query = "select id, example(name) as name, age from fake"
  6.   }
  7. }

Then the data in result table fake1 will update to

id name age
1 UDF: Joy Ding 20
2 UDF: May Ding 21
3 UDF: Kin Dom 24
4 UDF: Joy Dom 22

Changelog

new version

  • Add UDF of SQL Transform Connector