-
Flink Connector
Flink Connector Apache Flink supports creating Iceberg table directly without creating the explicit Flink catalog in Flink SQL. That means we can just create an iceberg table by s... -
Branching and Tagging
Branching and Tagging Overview Iceberg table metadata maintains a snapshot log, which represents the changes applied to a table. Snapshots are fundamental in Iceberg as they are ... -
Evolution
Evolution Iceberg supports in-place table evolution . You can evolve a table schema just like SQL — even in nested structures — or change partition layout when data volume chang... -
Publishing Data to S3
Introduction Hadoop and S3 The s3a File System The s3 File System Getting Gobblin to Publish to S3 Signing Up For AWS Setting Up EC2 Launching an EC2 Instance EC2 Package I... -
Kafka-HDFS Ingestion
Getting Started Standalone MapReduce Job Constructs Source and Extractor Writer and Publisher Job Config Properties Metrics and Events Task Level Metrics Task Level Events ... -
Google BigQuery
Querying from Google BigQuery Iceberg tables To read an Apache XTable™ (Incubating) synced Iceberg table from BigQuery , you have two options: Using Iceberg JSON metadata file ... -
Source schema and Converters
Source schema Converters Converters available in Gobblin Schema specification Supported data types by different converters Primitive types Complex types Array Map Record En... -
Client-Side Caching
Native implementation Client Side caching is implemented using client tracking listener through RESP3 protocol available in Redis or Valkey. It’s used to speed up read operation... -
Flink DDL
DDL commands CREATE Catalog Hive catalog This creates an Iceberg catalog named hive_catalog that can be configured using 'catalog-type'='hive' , which loads tables from Hive m... -
Java Custom Catalog
Custom Catalog It’s possible to read an iceberg table either from an hdfs path or from a hive table. It’s also possible to use a custom metastore in place of hive. The steps to do...
