Custom Catalog It’s possible to read an iceberg table either from an hdfs path or from a hive table. It’s also possible to use a custom metastore in place of hive. The steps to do...
Introduction Record format Configuration General configuration values Authentication No credentials Using certificates Using bucket password Document level expiration 1 - Ex...
Evolution Iceberg supports in-place table evolution . You can evolve a table schema just like SQL — even in nested structures — or change partition layout when data volume chang...
Over the years, LinkedIn’s data infrastructure team built custom solutions for ingesting diverse data entities into our Hadoop eco-system. At one point, we were running 15 t...
Description Key Features Options url [string] region [string] accessKeyId [string] secretAccessKey [string] table [string] common options Example Changelog next version ...
Id generator Redis or Valkey based Java Id generator RIdGenerator generates unique numbers but not monotonically increased. At first request, batch of id numbers is allocated and...
Native implementation Client Side caching is implemented using client tracking listener through RESP3 protocol available in Redis or Valkey. It’s used to speed up read operation...