MaxCompute - Enhances Data Writes to OSS External Tables
Feb 08 2021MaxCompute
Target customers: developers, users who want to export data, and users who use third-party engines to read exported data. Features released: If you perform INSERT operations to write data to OSS external tables, MaxCompute allows you to use the multipart upload feature of OSS to make data writes more efficient. In addition, the data written from MaxCompute to an OSS external table is stored in the .odps folder in the LOCATION directory, and a .meta file is maintained to ensure data consistency. The data in the .odps folder can be handled only by MaxCompute. If the data is read by other engines, errors may occur. The odps.sql.unstructured.oss.commit.mode attribute is added to MaxCompute. The default value is false. If the value is true, MaxCompute uses the multipart upload feature to write data and two-phase commit to ensure data consistency. In this case, no .odps directories or .meta files are generated. This allows other data processing engines to read the data.