-
Source schema and Converters
Source schema Converters Converters available in Gobblin Schema specification Supported data types by different converters Primitive types Complex types Array Map Record En... -
Logic
and isEmpty not or cond ifElse allPass anyPass defaultTo both complement either propSatisfies unless when and a → b → a | b Added in v0.1.0 Returns the first a... -
Evolution
Evolution Iceberg supports in-place table evolution . You can evolve a table schema just like SQL — even in nested structures — or change partition layout when data volume chang... -
Logic
and isEmpty not or cond ifElse allPass anyPass defaultTo both complement either and a → b → a | b Added in v0.1.0 Returns the first argument if it is falsy, other... -
Flink Connector
Flink Connector Apache Flink supports creating Iceberg table directly without creating the explicit Flink catalog in Flink SQL. That means we can just create an iceberg table by s... -
Deployment
Gobblin Execution Modes Overview One important feature of Gobblin is that it can be run on different platforms. Currently, Gobblin can run in standalone mode (which runs on a sing... -
Docker Integration
Introduction Docker Docker Repositories Run the docker image with simple wikipedia jobs Use Gobblin Standalone on Docker for Kafka and HDFS Ingestion Run Gobblin as a Service ... -
Hive Distcp
Introduction Configure Hive Distcp Job Source and target metastores Database and tables to copy Target path computation Conflicting table and partitions treatment Deregisterin... -
Writing ORC Data
Introduction Hive SerDe Integration Writing to an ORC File Data Flow Extending Gobblin’s SerDe Integration Introduction Gobblin is capable of writing data to ORC files by le... -
Home
Over the years, LinkedIn’s data infrastructure team built custom solutions for ingesting diverse data entities into our Hadoop eco-system. At one point, we were running 15 t...
