This topic describes the latest updates to MaxCompute documentation in 2021. These updates provide detailed information about the new features, new syntax, and permission updates in MaxCompute to help you improve the efficiency of project development.

Updates in December 2021

Release date Feature Category Description References
2021-12-24 External data source management New description External data sources can be created to connect to Hadoop clusters. This helps implement the lakehouse solution of MaxCompute. Manage external data sources
2021-12-22 Lakehouse solution of MaxCompute Updated description The lakehouse solution of MaxCompute can be implemented by using the DataWorks console. Lakehouse of MaxCompute
2021-12-06 Data visualization Updated description A new topic about data visualization on an online operational analytics platform is provided. Display data on dashboards

Updates in November 2021

Release date Feature Category Description References
2021-11-29 Policy-based permission management New practice Policy-based permission management is supported for users assigned built-in roles. Policy-based permission management for users assigned built-in roles
2021-11-29 Tenant-level user management Updated description Users can be managed in the MaxCompute console. Role planning and management
2021-11-15 Conversion of IP addresses to geolocations New practice User-defined functions (UDFs) can be used in MaxCompute to convert IPv4 or IPv6 addresses to geolocations. Use a MaxCompute UDF to convert IPv4 or IPv6 addresses into geolocations
2021-11-10 Python program upload and Python UDF creation New description After a Python program is developed and debugged, the Python program must be uploaded to a MaxCompute project as a resource. Then, the Python program can be used to create a MaxCompute UDF. Upload a Python program and create a MaxCompute UDF
2021-11-10 Visualized function management in MaxCompute Studio New description A visual editor is provided by Project Explorer of MaxCompute Studio. The visual editor helps you create UDFs in a MaxCompute project. The editor also provides an entry point that allows you to view the code of MaxCompute UDFs and delete the UDFs. Manage functions in a visualized manner
2021-11-08 Input parameters of date functions, mathematical functions, window functions, and aggregate functions Updated description The description for null values of input parameters of date functions, mathematical functions, window functions, and aggregate functions is added.
2021-11-03 Clearance of partition data in partitioned tables New description MaxCompute allows you to clear partition data that meet a specified filter condition. If you want to delete one or more partitions that meet a filter condition at a time, you can use an expression to specify the condition, use the condition to match partitions, and clear the partition data at a time. Clear data from a partition
2021-11-02 Visualized resource management in MaxCompute Studio New description A visual editor is provided by Project Explorer of MaxCompute Studio. The visual editor helps you add the resources that are required to create UDFs in a MaxCompute project. The resources include tables, JAR files, Python scripts, and compressed packages. The editor also provides an entry point that allows you to obtain or remove resources. Manage resources in a visualized manner

Updates in October 2021

Release date Feature Category Description References
2021-10-26 Clearance of partition data in partitioned tables New description Partition data that meets a specified filter condition can be cleared in MaxCompute. If you want to delete one or more partitions that meet a filter condition at a time, you can use an expression to specify the condition, use the condition to match partitions, and then clear the partition data. Partition and column operations
2021-10-19 Type conversions of date values New practice Data types can be converted among STRING, TIMESTAMP, and DATETIME. Convert data types among STRING, TIMESTAMP, and DATETIME

Updates in September 2021

Release date Feature Category Description References
2021-09-30 Content adjustment of MaxCompute UDFs Updated description The directory and content of MaxCompute UDFs are reconstructed based on user-defined scalar functions, user-defined table-valued functions (UDTFs), and user-defined aggregate functions (UDAFs). Examples and FAQ are added. Overview
2021-09-22 Change in the network connection process Updated description Some operations in the network connection process are performed by submitting application forms instead of submitting tickets. Network connection process
2021-09-16 Guide on how to connect business intelligence (BI) tools to MaxCompute New description BI tools can be used to visualize analysis results of data from MaxCompute in charts.
2021-09-16 Guide on how to connect database management tools to MaxCompute New description Database management tools can be used to manage MaxCompute projects.
2021-09-15 DATE_SUB function New description The DATE_SUB function can be used to subtract a number of days from a date. The logic of the DATE_SUB function is opposite to that of the DATE_ADD function. DATE_SUB
2021-09-15 Spark in Java Updated description Examples on how to use Spark in Java are added.
2021-09-08 Concept hierarchy New description The concept hierarchy of MaxCompute is added. Concept hierarchy

Updates in August 2021

Release date Feature Category Description References
2021-08-26 CORR function New description The CORR function is added to mathematical functions. This function calculates the Pearson correlation coefficient for two columns of data. Mathematical functions
2021-08-20 Syntax updates for aggregate functions Updated description The "Syntax" section is added to the "Aggregate functions" topic. Aggregate functions
2021-08-19 HASH, COMPRESS, DECOMPRESS, NULLIF, FORMAT_NUMBER, SHA, SHA1, SHA2, and CRC32 added to other functions New description
  • HASH: calculates the hash value of the input parameters.
  • COMPRESS: uses the GZIP algorithm to compress input strings.
  • DECOMPRESS: uses the GZIP algorithm to decompress the input parameters of the BINARY type.
  • NULLIF: returns NULL if the values of expr 1 and expr 2 are the same. Otherwise, expr1 is returned.
  • FORMAT_NUMBER: converts a number into a string of the specified format.
  • SHA: calculates the SHA-1 hash value of a string or a binary value and returns a hexadecimal string.
  • SHA1: calculates the SHA-1 hash value of a string or a binary value and returns the value as a hexadecimal string.
  • SHA2: calculates the SHA-2 family hash value of a string or a binary value. SHA-224, SHA-256, SHA-384, and SHA-512 are supported.
  • CRC32: calculates the cyclic redundancy check (CRC) value of a string or binary value.
Other functions
2021-08-19 TO_MILLIS function New description The TO_MILLIS function is added to date functions. This function converts a date into a UNIX timestamp in milliseconds. Date functions
2021-08-18 Change in the minimum number of MaxCompute CUs Updated description The minimum number of MaxCompute compute units (CUs) that you must purchase is changed from 10 CUs to 50 CUs. Overview
2021-08-11 Commercial use of the UPDATE, DELETE, and MERGE INTO statements Updated description As of August 10, 2021, the public preview of the UPDATE, DELETE, and MERGE INTO statements of MaxCompute SQL ends. After the public preview ends, you are charged when you use these statements. Pay-as-you-go jobs that already use the UPDATE, DELETE, and MERGE INTO statements are charged after the public preview ends. If you did not use the UPDATE, DELETE, or MERGE INTO statement due to concerns about service stability in the public preview phase, you no longer need to worry about stability when you use these statements. MaxCompute provides the same guarantees for the availability and stability of the UPDATE, DELETE, and MERGE INTO statements as the guarantee for MaxCompute SQL.
2021-08-10 Functions of complex data types New feature MaxCompute can process data of complex data types, such as MAP, ARRAY, and STRUCT, and provides a variety of functions to support diversified data processing. Complex type functions
2021-08-04 ETL tools New description Extract, transform, and load (ETL) tools can be used to manage MaxCompute projects.
2021-08-02 IN/NOT IN SUBQUERY and SCALAR SUBQUERY Updated description IN/NOT IN SUBQUERY and SCALAR SUBQUERY support multi-column subqueries, which is a feature in PostgreSQL. Subqueries

Updates in July 2021

Release date Feature Category Description References
2021-07-16 Top scheduled SQL tasks that are managed by using a dynamic filter New feature If a top scheduled SQL task is found, the ID of the most recent instance on which the task runs is displayed. We recommend that you use a dynamic filter to check and optimize the related job to reduce resource consumption and improve the job running performance. Use MaxCompute Management
2021-07-15 Limits on packages Updated description The limits on packages are added. Use a package

Updates in June 2021

Release date Feature Category Description References
2021-06-07 Description of data import and data export by using Data Integration Updated description The description of data import and data export by using Data Integration is modified.

Updates in May 2021

Release date Feature Category Description References
2021-05-31 Descriptions related to preparation items Updated description The preparation procedure and related descriptions are modified. Overview
2021-05-31 Quick start Updated description A quick start guide is provided for each tool. Quick Start

Updates in March 2021

Release date Feature Category Description References
2021-03-25 MaxCompute error codes Updated description New information is added to the MaxCompute error code list.
2021-03-16 NOT NULL attribute for a specific column in the CREATE TABLE syntax Updated description The NOT NULL attribute can be configured in the CREATE TABLE syntax to specify that the values in a specific column cannot be NULL. Table operations
2021-03-16 FROM_JSON and TO_JSON functions New description The FROM_JSON and TO_JSON functions are added. You can use the FROM_JSON function to convert data in the JSON format into data of a data type that is supported by MaxCompute. You can also use this function to extract information from data in the JSON format and return data of the ARRAY, MAP, or STRUCT type based on the jsonStr and schema information. You can use the TO_JSON function to convert data in the ARRAY, MAP, or STRUCT format into data in the JSON format. String functions
2021-03-16 ACID semantics for transactional tables New feature New atomicity, consistency, isolation, durability (ACID) semantics for transactional tables are added based on the ACID semantics for concurrent write jobs. ACID semantics
2021-03-12 A periodic check for scheduled tasks that are continuously run with empty output and scheduled tasks with continuous full table scans New feature Top scheduled SQL tasks that are continuously run with empty output and top scheduled SQL tasks with continuous full table scans can be periodically checked in MaxCompute. If these tasks are found, the ID of the most recent instance on which the task runs is displayed. We recommend that you check and optimize the related jobs to reduce resource consumption. Use MaxCompute Management
2021-03-12 Column-level access control Updated description Column-level access control is supported. Permissions
2021-03-12 Enabling and disabling of download control based on property settings Updated description A property can be configured to enable or disable download control. To enable or disable download control, you must configure a property at the project level as the project owner or a user who is assigned the Super_Administrator role. Policy-based access control
2021-03-01 Data lakehouse New feature MaxCompute provides the lakehouse solution, which enables you to build a data management platform that combines data lakes and data warehouses. The lakehouse solution integrates the flexibility and diverse ecosystems of data lakes with the enterprise-class deployment of data warehouses. Lakehouse of MaxCompute

Updates in February 2021

Release date Feature Category Description References
2021-02-23 Example of how to use a Python 2 UDTF to read resources from MaxCompute New example An example of how to use a Python 2 UDTF to read resources from MaxCompute is added. Python 2 UDFs
2021-02-23 Example of how to use a Python 3 UDTF to read resources from MaxCompute New example An example of how to use a Python 3 UDTF to read resources from MaxCompute is added. Python 3 UDFs
2021-02-23 External files referenced by Spark jobs New description The description that Spark jobs reference external files is added. Set up a Spark on MaxCompute development environment
2021-02-20 Example of how to use a UDTF with LATERAL VIEW New example An example of how to use a UDTF with LATERAL VIEW is added. UDTF usage
2021-02-09 Deletion of user permissions New description The description for the deletion of user permissions is added. User planning and management
2021-02-07 Removal of the limit that only constants can be used in VALUES Updated description The limit that only constants can be used in VALUES is removed. VALUES
2021-02-07 Multipart upload of Object Storage Service (OSS) external tables New description The multipart upload feature of OSS is supported. This feature allows you to perform the INSERT operation to write data to OSS external tables. Export unstructured data to OSS
2021-02-07 ARRAY_INTERSECT and SORT_ARRAY functions New description The ARRAY_INTERSECT function is added to calculate the intersection of two arrays. The isAsc parameter is added to the SORT_ARRAY function. This parameter specifies whether to sort the given arrays in ascending or descending order. The default value of this parameter is True, which indicates that given arrays are sorted in ascending order. Other functions
2021-02-07 Filter expressions and the COUNT_IF function New description Filter expressions are supported. You can specify filter conditions for an aggregate function in a SELECT clause. This way, you can separately control the data range of the aggregate function in an aggregate statement. The COUNT_IF function is added to count the number of data records that meet the IF condition. Aggregate functions
2021-02-07 Support for the DATE and TIMESTAMP data types in DATEADD, DATE_ADD, DATEDIFF, DATEPART, and DATETRUNC functions New description The date functions DATEADD, DATE_ADD, DATEDIFF, DATEPART, and DATETRUNC are enhanced to support the DATE and TIMESTAMP data types. Date functions
2021-02-07 BASE64, UNBASE64, and PARSE_URL functions New description The string function PARSE_URL is added to parse URLs. The BASE64 and UNBASE64 functions are added to implement conversions between binary strings and Base64-encoded strings. String functions
2021-02-07 Disabling of header display in the MaxCompute command line to facilitate shell calls New description In a shell window or in the Windows command line, you may need to use the dynamic return value that is obtained by executing the odpscmd -e SQL statement. The variable of the shell can obtain this dynamic return value and run subsequent jobs in the shell. In this scenario, only field values are required. Other information, such as runtime information and headers, must not be returned. You can run set odps.sql.select.output.format={needHeader:false,fieldDelim:""}; to disable the header display and export the standard output from the calculation result to the destination handle. MaxCompute client
2021-02-07 Data export from MaxCompute to OSS by using UNLOAD New feature MaxCompute allows you to export data to OSS by using the UNLOAD command. This way, you can use OSS to store structured data and use other computing engines in OSS to process and analyze the data that is exported from MaxCompute. UNLOAD
2021-02-07 SKEWJOIN HINT New feature If two tables that you want to join contain hot key values, a long tail issue may occur. You can remove the hot key value and divide the data into hot data and non-hot data, and then merge them to improve the join efficiency. In this case, SKEWJOIN HINT can be used to automatically or manually extract hot key values from the two tables, separately calculate the join results of the hot key values and non-hot key values, and then join the calculated data. This way, the JOIN operation is accelerated. SKEWJOIN HINT
2021-02-07 Reading and writing MC-Hologres data by using the JDBC driver New feature You can use MaxCompute external tables to access data of MC-Hologres data sources by using the Java Database Connectivity (JDBC) driver. You can create an MC-Hologres external table in MaxCompute to query the data of MC-Hologres data sources by using the JDBC driver that is provided by PostgreSQL. This method prevents redundant data storage and allows you to obtain query results at a fast speed without the need to import or export data. Reading and writing data of MC-Hologres external tables is in the public preview stage. Data computing is free of charge but the Service Level Agreement (SLA) is not guaranteed. Hologres external tables
2021-02-01 Change records of Mars V0.6.3 New description Change records of Mars V0.6.3 are added. Version updates

Updates in January 2021

Release date Feature Category Description References
2021-01-21 Change in the return behavior of the GET_JSON_OBJECT function Updated description For new MaxCompute projects, the GET_JSON_OBJECT function retains the original strings instead of escaping JSON reserved characters when the function returns a value. GET_JSON_OBJECT
2021-01-19 Content adjustment and optimization of the SELECT syntax Updated description The content of the SELECT syntax is adjusted, and sample statements are optimized. SELECT syntax
2021-01-05 Description of data migration New description The Data migration topic is added. The content related to data upload and download and the migration tool is moved under this topic. Data upload and download