-
IBM Cloud
IBM COS configs IBM Cloud Object Storage Credentials IBM Cloud Object Storage Libs In this page, we explain how to get your Hudi spark job to store into IBM Cloud Object Storag... -
Collection
_.countBy(collection, [iteratee=_.identity]) Since Arguments Returns Example _.every(collection, [predicate=_.identity]) Since Arguments Returns Example _.filter(collec... -
Configure Kyuubi to use Custom Authentication
Build A Custom Authenticator Enable Custom Authentication Besides the builtin authentication methods, kyuubi supports custom authentication implementations of org.apache.kyuubi... -
Writes
Spark Writes To use Iceberg in Spark, first configure Spark catalogs . Some plans are only available when using Iceberg SQL extensions in Spark 3. Iceberg uses Apache Spark’s D... -
Retention
Introduction Design Overview of Gobblin Config Management Library Retention Constructs DatasetCleaner DatasetFinder ManagedCleanableDatasetFinder ConfigurableCleanableDataset ... -
Schedulers
Introduction Quartz Azkaban Oozie Launching Gobblin in Local Mode Example Config Files Uploading Files to HDFS Adding Gobblin jar Dependencies Launching the Job Launching ... -
Flink Queries
Flink Queries Iceberg support streaming and batch read With Apache Flink ‘s DataStream API and Table API. Reading with SQL Iceberg support both streaming and batch read in Flink... -
Introduction
Documentation Apache Iceberg is an open table format for huge analytic datasets. Iceberg adds tables to compute engines including Spark, Trino, PrestoDB, Flink, Hive and Impala u... -
Batch Reads
Spark DataSource API Daft Spark DataSource API The hudi-spark module offers the DataSource API to read a Hudi table into a Spark DataFrame. A time-travel query example: val ... -
Chaining
_(value) Arguments Returns Example _.chain(value) Arguments Returns Example _.tap(value, interceptor) Arguments Returns Example _.prototype.chain() Returns Example _...
