-
Writing ORC Data
Introduction Hive SerDe Integration Writing to an ORC File Data Flow Extending Gobblin’s SerDe Integration Introduction Gobblin is capable of writing data to ORC files by le... -
Template
Overview How to Use Templates Available Templates How to Create Your Own Template How does Template Work in Gobblin Overview The job configuration template is implemented fo... -
Transactions
Transactions management RMap , RMapCache , RLocalCachedMap , RSet , RSetCache and RBucket Redisson objects may participant in Transaction with ACID properties. It uses locks for... -
Gobblin Metrics Performance
Generalities How to interpret these numbers What if I need larger QPS? Update Metrics Performance Multiple metric updates per iteration Multi-threading Running Performance Tes... -
Parquet HDFS
Description Usage Example Pipeline Configuration Configuration Developer Notes Description An extension to FsDataWriter that writes in Parquet format in the form of either... -
Gobblin Compliance Design
Introduction Design Onboarding Purger Gobblin constructs Hive operations Retention Restore Introduction The Gobblin Compliance module allows for data purging to meet reg... -
EventMesh Schema Registry (OpenSchema)
EventMesh Schema Registry (OpenSchema) Overview of Schema and Schema Registry Schema Schema Registry Comparison of Schema Registry in Different Projects Overview of OpenSchema ... -
High Level Consumer
Problem Statement New Design & Details Problem Statement Current Gobblin Kafka High Level Consumer uses Kafka Consumer (0.8) APIs and gobblin support for them will be depreca... -
Server-Side Scripting
LUA Scripting Redisson provides RScript object to execute Lua script. It has atomicity property and used to process data on Redis or Valkey side. Script could be executed in two ... -
Flink Actions
Rewrite files action Iceberg provides API to rewrite small files into large files by submitting Flink batch jobs. The behavior of this Flink action is the same as Spark’s rewriteD...
