Apache Paimon is a unified stream and batch lake storage format that supports high-throughput writes and low-latency queries. E-MapReduce compute engines such as Flink, Spark, Hive, and Trino integrate well with Paimon, allowing you to build data lake storage on HDFS or OSS for data lake analytics.
Apache Paimon provides the following core features:
-
Low-cost, lightweight data lake storage services on HDFS or OSS.
-
Reading and writing large-scale datasets in both stream and batch modes.
-
Batch and OLAP queries with data freshness ranging from minutes to seconds.
-
Incremental data consumption and production, serving as a storage layer for both traditional and streaming data warehouses.
-
Data pre-aggregation to reduce storage costs and the computational load on downstream systems.
-
Time travel to query historical data versions.
-
Efficient data filtering.
-
Schema evolution for tables.
For more information, see Apache Paimon.