-
Troubleshooting
Writing Tables org.apache.parquet.io.InvalidRecordException: Parquet/Avro schema mismatch: Avro field ‘col1’ not found java.lang.UnsupportedOperationException: org.apache.parquet.... -
Clustering
Background How is compaction different from clustering? Clustering Architecture Overall, there are 2 steps to clustering Schedule clustering Execute clustering Clustering Use... -
Design & Concepts
How does Hudi ensure atomicity? Does Hudi extend the Hive table layout? What concurrency control approaches does Hudi adopt? Hudi’s commits are based on transaction start time i... -
Deployment
Deploying Hudi Streamer Spark Datasource Writer Jobs Upgrading Downgrading Migrating This section provides all the help you need to deploy and operate Hudi tables at scale. ... -
Indexing
Indexing Multi-modal Indexing Index Types in Hudi Global and Non-Global Indexes Configs Spark based configs Flink based configs Indexing Strategies Workload 1: Late arriving... -
Creating your first interoperable table
Pre-requisites Steps Initialize a pyspark shell Create dataset Running sync Conclusion Next steps Using OneTable to sync your source tables in different target format invo... -
Hive Metastore
Spark Data Source example Query using HiveQL Use partition extractor properly Hive Sync Tool Hive Sync Configuration Sync modes HMS JDBC HIVEQL Flink Setup Install Hive E... -
Google BigQuery
Sync Modes Manifest File Benefits of using the new manifest approach: View Over Files (Legacy) Configurations Partition Handling Example Hudi tables can be queried from Goo... -
Table Services
What does the Hudi cleaner do? How do I run compaction for a MOR table? What options do I have for asynchronous/offline compactions on MOR table? How to disable all table servic... -
Spark Tuning Guide
Spark Tuning Guide Writing General Tips Spark failures Hudi consumes too much space in a temp folder while upsert How to tune shuffle parallelism of Hudi jobs ? GC Tuning ...
