-
Streaming Reads
Spark Streaming Spark Streaming Structured Streaming reads are based on Hudi’s Incremental Query feature, therefore streaming read can return data for which commits and base fil... -
Relation
countBy difference differenceWith unionWith union gt gte intersection lt lte max min propEq sortBy countBy (a → String) → [a] → {*} Parameters fnThe function u... -
第四章：线程安全问题
学习目标 1. 什么是线程安全 1.1 线程安全的定义 1.2 线程安全的重要性 2. 共享资源访问的竞态条件 2.1 什么是竞态条件 2.2 竞态条件示例 2.3 竞态条件的类型 读-改-写竞态条件 检查再执行竞态条件 3. 线程安全问题的表现形式 3.1 原子性问题 3.2 可见性问题 3.3 有序性问题 4. 实战案例：商品... -
Writes
Spark Writes To use Iceberg in Spark, first configure Spark catalogs . Some plans are only available when using Iceberg SQL extensions in Spark 3. Iceberg uses Apache Spark’s D... -
第六章：volatile关键字
学习目标 1 内存可见性问题 1.1 什么是内存可见性 1.2 内存可见性问题的原因 2 volatile的作用与使用场景 2.1 volatile关键字介绍 2.2 volatile的内存语义 2.3 volatile的适用场景 2.4 volatile不能解决的问题 3 volatile与synchronized的区别 3.1 可见性与... -
Console
Support Connector Version Support Those Engines Description Key Features Options Task Example Simple: Multiple Sources Simple: Console Sample Data Console sink connector... -
Object
clone values eqProps keys omit pick pickAll project prop props where keysIn path valuesIn clone {*} → {*} Parameters valueThe object or array to clone Return... -
Branching and Tagging
Branching and Tagging Overview Iceberg table metadata maintains a snapshot log, which represents the changes applied to a table. Snapshots are fundamental in Iceberg as they are ... -
Distributed objects
Object holder Java implementation of Redis or Valkey based RBucket object is a holder for any type of object. Size is limited to 512Mb. Code example: RBucket < AnyObject > buc... -
Structured Streaming
Spark Structured Streaming Iceberg uses Apache Spark’s DataSourceV2 API for data source and catalog implementations. Spark DSv2 is an evolving API with different levels of support...
