This topic describes the ACID semantics of MaxCompute concurrent jobs.
Terms
- Operation: a single job submitted in MaxCompute.
- Data object: an object that contains data, such as a partitioned or non-partitioned table.
- INTO job: an SQL job that contains the INTO keyword, such as INSERT INTO or DYNAMIC INSERT INTO.
- OVERWRITE job: an SQL job that contains the OVERWRITE keyword, such as INSERT OVERWRITE or DYNAMIC INSERT OVERWRITE.
- Data upload by using Tunnel: an INTO or OVERWRITE job.
Description of ACID semantics
- Atomicity: An operation is either fully complete or not executed at all, namely, an operation is not performed partially.
- Consistency: The integrity of data objects is maintained when an operation is performed.
- Isolation: An operation is complete independent of other concurrent operations.
- Durability: After an operation is complete, modified data is permanently valid and not lost even if a system failure occurs.
Scenarios of ACID semantics for MaxCompute
- Atomicity
- When multiple jobs conflict with each other, MaxCompute ensures that only one job succeeds.
- The atomicity of the CREATE, OVERWRITE, and DROP operations on a single table or partition is ensured.
- The atomicity of cross-table operations such as MULTI-INSERT cannot be ensured.
- In extreme cases, the following operations may not be atomic:
- A
DYNAMIC INSERT OVERWRITE
operation that is performed on more than 10,000 partitions. - An INTO operation. INTO operations cannot ensure atomicity because data cleansing fails during a transaction rollback. However, the transaction rollback does not cause raw data loss.
- A
- Consistency
- OVERWRITE jobs ensure consistency.
- If an INTO job fails due to a conflict, data from the failed job may remain.
- Isolation
- Non-INTO operations ensure that read operations are committed.
- In INTO operations, some read operations may not be committed.
- Durability
- MaxCompute ensures data durability.