-
Retention
Introduction Design Overview of Gobblin Config Management Library Retention Constructs DatasetCleaner DatasetFinder ManagedCleanableDatasetFinder ConfigurableCleanableDataset ... -
Home
Over the years, LinkedIn’s data infrastructure team built custom solutions for ingesting diverse data entities into our Hadoop eco-system. At one point, we were running 15 t... -
Kafka
Introduction Pre-requisites Steps Configuration Details What Next? Introduction The Kafka writer allows users to create pipelines that ingest data from Gobblin sources into ... -
Source schema and Converters
Source schema Converters Converters available in Gobblin Schema specification Supported data types by different converters Primitive types Complex types Array Map Record En... -
Monitoring
Overview Metrics Collecting and Reporting Metrics Reporting Metrics collection JVM Metrics Pre-defined Job Execution Metrics Job Execution History Store Email Notifications ... -
Connectors for Hive SQL Query Engine
Apache Paimon (Incubating) Iceberg -
Google Webmaster
Introduction Implementation Summary Entities Work Flow Configuration Introduction The Google Search Console data ingestion project is to download query and analytics data f... -
RisingWave
RisingWave RisingWave is a Postgres-compatible SQL database designed for real-time event streaming data processing, analysis, and management. It can ingest millions of events per... -
Glue Data Catalog
Syncing to Glue Data Catalog This document walks through the steps to register an Apache XTable™ (Incubating) synced table in Glue Data Catalog on AWS. Pre-requisites Source ta... -
Partitioning
Partitioning What is partitioning? Partitioning is a way to make queries faster by grouping similar rows together when writing. For example, queries for log entries from a logs ...
