Support Those Engines Key features Description Sink Options Task Example Changelog 2.2.0-beta 2022-09-26 Socket sink connector Support Those Engines Spark Flink SeaTu...
Disclaimer Supported Storage System Verified Combination of Spark and storage system HDInsight Spark2.4 on Azure Data Lake Storage Gen 2 Databricks Spark2.4 on Azure Data Lake S...
Spark Structured Streaming Iceberg uses Apache Spark’s DataSourceV2 API for data source and catalog implementations. Spark DSv2 is an evolving API with different levels of support...
Branching and Tagging Overview Iceberg table metadata maintains a snapshot log, which represents the changes applied to a table. Snapshots are fundamental in Iceberg as they are ...
Spark Streaming Spark Streaming Structured Streaming reads are based on Hudi’s Incremental Query feature, therefore streaming read can return data for which commits and base fil...
Rewrite files action. Rewrite files action. Iceberg provides API to rewrite small files into large files by submitting Flink batch jobs. The behavior of this Flink action is the...
Support Connector Version Support Those Engines Description Key Features Options Task Example Simple: Multiple Sources Simple: Console Sample Data Console sink connector...
Debugging Server Debugging Engine Flink Engine Trino Engine Hive Engine Debugging Apps Spark Engine Flink Engine You can use the Java Debug Wire Protocol to debug Kyuubi ...