-
Structured Streaming
Spark Structured Streaming Iceberg uses Apache Spark’s DataSourceV2 API for data source and catalog implementations. Spark DSv2 is an evolving API with different levels of support... -
Job Execution History Store
Overview Information Recorded Job Execution Information Task Execution Information Default Implementation Rest Query API Example Queries Job Execution History Server Over... -
Event Logging
Event Logging StatsD Logging Event Logging Superset by default logs special action events in its internal database (DBEventLogger). These logs can be accessed on the UI by navi... -
The AGE Cypher Query Format
Cypher() Cypher in an Expression SELECT Clause Cypher queries are constructed using a function called cypher in ag_catalog which returns a Postgres SETOF records . Cypher() c... -
Building Gobblin
Overview Information Recorded Job Execution Information Task Execution Information Default Implementation Rest Query API Example Queries Job Execution History Server Over... -
Hudi
Hudi Integration Dependencies Hudi Operations Apache Hudi (pronounced “hoodie”) is the next generation streaming data lake platform. Apache Hudi brings core warehouse and datab... -
Object
clone values eqProps keys omit pick pickAll project prop props keysIn path valuesIn toPairs toPairsIn propOr has hasIn assoc assocPath lens pickBy evolve inv... -
Hudi
Hudi Integration Dependencies Configurations Hudi Operations Apache Hudi (pronounced “hoodie”) is the next generation streaming data lake platform. Apache Hudi brings core war... -
Branching and Tagging
Branching and Tagging Overview Iceberg table metadata maintains a snapshot log, which represents the changes applied to a table. Snapshots are fundamental in Iceberg as they are ... -
Solution for Big Result Sets
Incremental collection Use in single connections Change incremental collection mode in session Typically, when a user submits a SELECT query to Spark SQL engine, the Driver cal...
