This topic describes the ACID semantics of MaxCompute for concurrent jobs.

Terms

  • Operation: a single job submitted in MaxCompute
  • Data object: an object that contains data, such as a partitioned or non-partitioned table
  • INTO job: an SQL job that contains the INTO keyword, such as INSERT INTO or DYNAMIC INSERT INTO
  • OVERWRITE job: an SQL job that contains the OVERWRITE keyword, such as INSERT OVERWRITE or DYNAMIC INSERT OVERWRITE
  • Data upload by using Tunnel: an INTO or OVERWRITE job

Descriptions of ACID semantics

  • Atomicity: An operation is either fully completed or not executed at all.
  • Consistency: The integrity of data objects is maintained while operations are executed.
  • Isolation: An operation is completed independent of other concurrent operations.
  • Durability: After an operation is completed, modified data is permanently valid and not lost even if a system failure occurs.

Scenarios of ACID semantics for MaxCompute

  • Atomicity
    • When multiple jobs conflict with each other, MaxCompute always ensures that only one job succeeds.
    • The atomicity of the CREATE, OVERWRITE, and DROP operations on a single table or partition is guaranteed.
    • Atomicity cannot be guaranteed for cross-table operations such as MULTI-INSERT.
    • In extreme cases, the following operations may not be atomic:
      • The DYNAMIC INSERT OVERWRITE operation is performed on more than 10,000 partitions.
      • An INTO operation fails because data cleansing fails during a transaction rollback. However, this does not cause raw data loss.
  • Consistency
    • OVERWRITE jobs ensure consistency.
    • If an INTO job fails due to a conflict, data from the failed job may remain.
  • Isolation
    • Non-INTO operations ensure that read operations are committed.
    • In INTO operations, some read operations may not be committed.
  • Durability
    • MaxCompute ensures data durability.