What is Delta table, basic concepts - MaxCompute - Alibaba Cloud Documentation Center

MaxCompute Delta table is an incremental data table format introduced by MaxCompute. It enables minute-level data import, uses AliORC as the underlying file format, and supports UPSERT semantics. It also provides the ability to read and write incremental data in a Change Data Capture (CDC)-like format. MaxCompute Delta table is deeply integrated with the backend storage and metadata services of MaxCompute, automatically optimizing the governance of table data files to reduce storage costs and improve read/write efficiency. This topic provides an overview of the foundational design principles of MaxCompute Delta table.

Feature	Description
Delta table format	A new incremental table data format that supports efficient storage and read/write operations of both incremental and full data. It also supports primary key settings and commonly used near-real-time data warehousing functionalities, such as real-time data updates. For more information, see Table data format.
Near-real-time incremental import	Near-real-time streaming data writes: MaxCompute collaborates with related products to develop a variety of data access tools, such as the custom Flink connector for MaxCompute and the Data Integration service of DataWorks. These tools support high-concurrency, scalable, and efficient near-real-time incremental data import. Batch writes of full and incremental data: MaxCompute SQL and other interfaces can be used to efficiently batch write incremental and full data.
Data organization optimization	The storage service of MaxCompute manages incremental data files in an intelligent manner. The management of incremental data files includes optimization operations, such as clustering of small files, data compaction, and data sorting. The storage service automatically executes data optimization tasks for some operations based on a comprehensive evaluation of multiple dimensions, such as data characteristics and time series, to help maintain stable and efficient data storage and computing.
Incremental queries	SQL syntax can be used to query incremental data. For more information, see Incremental queries.
Time travel	SQL syntax can be used to query historical snapshots. This helps trace back the historical state of business data and recover incorrect data. For more information, see Time travel.
Transaction management	This feature supports transaction concurrency conflict management, data version management, time travel management, and metadata update and analytics in incremental data processing scenarios.