RisingWave

RisingWave is a Postgres-compatible SQL database designed for real-time event streaming data processing, analysis, and management. It can ingest millions of events per second, continuously join and analyze live data streams with historical tables, serve ad-hoc queries in real-time, and deliver fresh, consistent results.

Supported Features

RisingWave supports batch reads and streaming writes of Apache Iceberg™ tables via its built-in source and sink connectors. For more information, see the Iceberg source connector documentation and Iceberg sink connector documentation.

Table Formats and Warehouse Locations

Currently, RisingWave only supports the Iceberg V2 table format and S3-compatible object storage as Iceberg warehouse locations.

Catalogs

RisingWave supports the following catalogs:

  • rest
  • jdbc / sql
  • glue
  • storage
  • hive

See RisingWave’s Iceberg catalog documentation for more details.

Getting Started

Writing Data to Iceberg Tables

To write data to an Iceberg table, create a sink in RisingWave. The following example writes data from an existing table or materialized view rw_data to an Iceberg table t1.

  1. CREATE SINK sink_to_iceberg FROM t1 WITH (
  2.     connector = 'iceberg',
  3.     type = 'upsert',
  4.     primary_key = 'id',
  5.     database.name = 'demo_db',
  6.     table.name = 't1',
  7.     catalog.name = 'demo',
  8.     catalog.type = 'storage',
  9.     warehouse.path = 's3a://hummock001/demo',
  10.     s3.endpoint = '<http://127.0.0.1:9301>',
  11.     s3.region = 'us-east-1',
  12.     s3.access.key = 'hummockadmin',
  13.     s3.secret.key = 'hummockadmin'
  14. );

Note: From RisingWave 2.1, you can use the create_table_if_not_exists parameter to create a table if it doesn’t exist.

Reading from Iceberg Tables

To read data from an Iceberg table, create a source in RisingWave. The following example reads data from an Iceberg table t1.

  1. CREATE SOURCE iceberg_t1_source WITH (
  2.     connector = 'iceberg',
  3.     s3.endpoint = '<http://127.0.0.1:9301>',
  4.     s3.region = 'us-east-1',
  5.     s3.access.key = 'hummockadmin',
  6.     s3.secret.key = 'hummockadmin',
  7.     s3.path.style.access = 'true',
  8.     catalog.type = 'storage',
  9.     warehouse.path = 's3a://hummock001/demo',
  10.     database.name = 'demo_db',
  11.     table.name = 't1',
  12. );

After this source is created, you can query the data using the following SQL statement: 

  1. SELECT * FROM iceberg_t1_source;