This topic describes the latest updates to MaxCompute documentation in 2021. These updates provide detailed information about the new features, new syntax, and permission updates in MaxCompute to help you improve the efficiency of project development.
Updates in December 2021
Release date | Feature | Category | Description | References |
2021-12-22 | Lakehouse solution of MaxCompute | Updated description | The lakehouse solution of MaxCompute can be implemented by using the DataWorks console. |
Updates in November 2021
Release date | Feature | Category | Description | References |
2021-11-29 | Policy-based permission management | New practice | Policy-based permission management is supported for users assigned built-in roles. | Policy-based permission management for users assigned built-in roles |
2021-11-29 | Tenant-level user management | Updated description | Users can be managed in the MaxCompute console. | |
2021-11-15 | Conversion of IP addresses to geolocations | New practice | User-defined functions (UDFs) can be used in MaxCompute to convert IPv4 or IPv6 addresses to geolocations. | Use a MaxCompute UDF to convert IPv4 or IPv6 addresses into geolocations |
2021-11-10 | Python program upload and Python UDF creation | New description | After a Python program is developed and debugged, the Python program must be uploaded to a MaxCompute project as a resource. Then, the Python program can be used to create a MaxCompute UDF. | |
2021-11-10 | Visualized function management in MaxCompute Studio | New description | A visual editor is provided by Project Explorer of MaxCompute Studio. The visual editor helps you create UDFs in a MaxCompute project. The editor also provides an entry point that allows you to view the code of MaxCompute UDFs and delete the UDFs. | |
2021-11-08 | Input parameters of date functions, mathematical functions, window functions, and aggregate functions | Updated description | The description for null values of input parameters of date functions, mathematical functions, window functions, and aggregate functions is added. | |
2021-11-03 | Clearance of partition data in partitioned tables | New description | MaxCompute allows you to clear partition data that meet a specified filter condition. If you want to delete one or more partitions that meet a filter condition at a time, you can use an expression to specify the condition, use the condition to match partitions, and clear the partition data at a time. | |
2021-11-02 | Visualized resource management in MaxCompute Studio | New description | A visual editor is provided by Project Explorer of MaxCompute Studio. The visual editor helps you add the resources that are required to create UDFs in a MaxCompute project. The resources include tables, JAR files, Python scripts, and compressed packages. The editor also provides an entry point that allows you to obtain or remove resources. |
Updates in October 2021
Release date | Feature | Category | Description | References |
2021-10-26 | Clearance of partition data in partitioned tables | New description | Partition data that meets a specified filter condition can be cleared in MaxCompute. If you want to delete one or more partitions that meet a filter condition at a time, you can use an expression to specify the condition, use the condition to match partitions, and then clear the partition data. | |
2021-10-19 | Type conversions of date values | New practice | Data types can be converted among STRING, TIMESTAMP, and DATETIME. |
Updates in September 2021
Release date | Feature | Category | Description | References |
2021-09-30 | Content adjustment of MaxCompute UDFs | Updated description | The directory and content of MaxCompute UDFs are reconstructed based on user-defined scalar functions, user-defined table-valued functions (UDTFs), and user-defined aggregate functions (UDAFs). Examples and FAQ are added. | |
2021-09-22 | Change in the network connection process | Updated description | Some operations in the network connection process are performed by submitting application forms instead of submitting tickets. | |
2021-09-16 | Guide on how to connect business intelligence (BI) tools to MaxCompute | New description | BI tools can be used to visualize analysis results of data from MaxCompute in charts. | |
2021-09-16 | Guide on how to connect database management tools to MaxCompute | New description | Database management tools can be used to manage MaxCompute projects. | |
2021-09-15 | DATE_SUB function | New description | The DATE_SUB function can be used to subtract a number of days from a date. The logic of the DATE_SUB function is opposite to that of the DATE_ADD function. | |
2021-09-15 | Spark in Java | Updated description | Examples on how to use Spark in Java are added. | |
2021-09-08 | Concept hierarchy | New description | The concept hierarchy of MaxCompute is added. |
Updates in August 2021
Release date | Feature | Category | Description | References |
2021-08-26 | CORR function | New description | The CORR function is added to mathematical functions. This function calculates the Pearson correlation coefficient for two columns of data. | |
2021-08-20 | Syntax updates for aggregate functions | Updated description | The "Syntax" section is added to the "Aggregate functions" topic. | |
2021-08-19 | HASH, COMPRESS, DECOMPRESS, NULLIF, FORMAT_NUMBER, SHA, SHA1, SHA2, and CRC32 added to other functions | New description |
| |
2021-08-19 | TO_MILLIS function | New description | The TO_MILLIS function is added to date functions. This function converts a date into a UNIX timestamp in milliseconds. | |
2021-08-18 | Change in the minimum number of MaxCompute CUs | Updated description | The minimum number of MaxCompute compute units (CUs) that you must purchase is changed from 10 CUs to 50 CUs. | |
2021-08-11 | Commercial use of the UPDATE, DELETE, and MERGE INTO statements | Updated description | As of August 10, 2021, the public preview of the UPDATE, DELETE, and MERGE INTO statements of MaxCompute SQL ends. After the public preview ends, you are charged when you use these statements. Pay-as-you-go jobs that already use the UPDATE, DELETE, and MERGE INTO statements are charged after the public preview ends. If you did not use the UPDATE, DELETE, or MERGE INTO statement due to concerns about service stability in the public preview phase, you no longer need to worry about stability when you use these statements. MaxCompute provides the same guarantees for the availability and stability of the UPDATE, DELETE, and MERGE INTO statements as the guarantee for MaxCompute SQL. | |
2021-08-10 | Functions of complex data types | New feature | MaxCompute can process data of complex data types, such as MAP, ARRAY, and STRUCT, and provides a variety of functions to support diversified data processing. | |
2021-08-04 | ETL tools | New description | Extract, transform, and load (ETL) tools can be used to manage MaxCompute projects. | |
2021-08-02 |
| Updated description | IN/NOT IN SUBQUERY and SCALAR SUBQUERY support multi-column subqueries, which is a feature in PostgreSQL. |
Updates in July 2021
Release date | Feature | Category | Description | References |
2021-07-16 | Top scheduled SQL tasks that are managed by using a dynamic filter | New feature | If a top scheduled SQL task is found, the ID of the most recent instance on which the task runs is displayed. We recommend that you use a dynamic filter to check and optimize the related job to reduce resource consumption and improve the job running performance. | |
2021-07-15 | Limits on packages | Updated description | The limits on packages are added. |
Updates in June 2021
Release date | Feature | Category | Description | References |
2021-06-07 | Description of data import and data export by using Data Integration | Updated description | The description of data import and data export by using Data Integration is modified. |
Updates in May 2021
Release date | Feature | Category | Description | References |
2021-05-31 | Descriptions related to preparation items | Updated description | The preparation procedure and related descriptions are modified. | |
2021-05-31 | Quick start | Updated description | A quick start guide is provided for each tool. |
Updates in March 2021
Release date | Feature | Category | Description | References |
2021-03-25 | MaxCompute error codes | Updated description | New information is added to the MaxCompute error code list. | |
2021-03-16 | NOT NULL attribute for a specific column in the CREATE TABLE syntax | Updated description | The NOT NULL attribute can be configured in the CREATE TABLE syntax to specify that the values in a specific column cannot be NULL. | |
2021-03-16 | FROM_JSON and TO_JSON functions | New description | The FROM_JSON and TO_JSON functions are added. You can use the FROM_JSON function to convert data in the JSON format into data of a data type that is supported by MaxCompute. You can also use this function to extract information from data in the JSON format and return data of the ARRAY, MAP, or STRUCT type based on the jsonStr and schema information. You can use the TO_JSON function to convert data in the ARRAY, MAP, or STRUCT format into data in the JSON format. | |
2021-03-16 | ACID semantics for transactional tables | New feature | New atomicity, consistency, isolation, durability (ACID) semantics for transactional tables are added based on the ACID semantics for concurrent write jobs. | |
2021-03-12 | A periodic check for scheduled tasks that are continuously run with empty output and scheduled tasks with continuous full table scans | New feature | Top scheduled SQL tasks that are continuously run with empty output and top scheduled SQL tasks with continuous full table scans can be periodically checked in MaxCompute. If these tasks are found, the ID of the most recent instance on which the task runs is displayed. We recommend that you check and optimize the related jobs to reduce resource consumption. | |
2021-03-12 | Column-level access control | Updated description | Column-level access control is supported. | |
2021-03-12 | Enabling and disabling of download control based on property settings | Updated description | A property can be configured to enable or disable download control. To enable or disable download control, you must configure a property at the project level as the project owner or a user who is assigned the Super_Administrator role. | |
2021-03-01 | Data lakehouse | New feature | MaxCompute provides the lakehouse solution, which enables you to build a data management platform that combines data lakes and data warehouses. The lakehouse solution integrates the flexibility and diverse ecosystems of data lakes with the enterprise-class deployment of data warehouses. |
Updates in February 2021
Release date | Feature | Category | Description | References |
2021-02-23 | Example of how to use a Python 2 UDTF to read resources from MaxCompute | New example | An example of how to use a Python 2 UDTF to read resources from MaxCompute is added. | |
2021-02-23 | Example of how to use a Python 3 UDTF to read resources from MaxCompute | New example | An example of how to use a Python 3 UDTF to read resources from MaxCompute is added. | |
2021-02-23 | External files referenced by Spark jobs | New description | The description that Spark jobs reference external files is added. | |
2021-02-20 | Example of how to use a UDTF with LATERAL VIEW | New example | An example of how to use a UDTF with LATERAL VIEW is added. | |
2021-02-09 | Deletion of user permissions | New description | The description for the deletion of user permissions is added. | |
2021-02-07 | Removal of the limit that only constants can be used in VALUES | Updated description | The limit that only constants can be used in VALUES is removed. | |
2021-02-07 | Multipart upload of Object Storage Service (OSS) external tables | New description | The multipart upload feature of OSS is supported. This feature allows you to perform the INSERT operation to write data to OSS external tables. | |
2021-02-07 |
| New description | The | |
2021-02-07 | Filter expressions and the | New description | Filter expressions are supported. You can specify filter conditions for an aggregate function in a | |
2021-02-07 | Support for the DATE and TIMESTAMP data types in | New description | The date functions | |
2021-02-07 |
| New description | The string function | |
2021-02-07 | Disabling of header display in the MaxCompute command line to facilitate shell calls | New description | In a shell window or in the Windows command line, you may need to use the dynamic return value that is obtained by executing the | |
2021-02-07 | Data export from MaxCompute to OSS by using UNLOAD | New feature | MaxCompute allows you to export data to OSS by using the UNLOAD command. This way, you can use OSS to store structured data and use other computing engines in OSS to process and analyze the data that is exported from MaxCompute. | |
2021-02-07 | SKEWJOIN HINT | New feature | If two tables that you want to join contain hot key values, a long tail issue may occur. You can remove the hot key value and divide the data into hot data and non-hot data, and then merge them to improve the join efficiency. In this case, SKEWJOIN HINT can be used to automatically or manually extract hot key values from the two tables, separately calculate the join results of the hot key values and non-hot key values, and then join the calculated data. This way, the JOIN operation is accelerated. | |
2021-02-07 | Reading and writing MC-Hologres data by using the JDBC driver | New feature | You can use MaxCompute external tables to access data of MC-Hologres data sources by using the Java Database Connectivity (JDBC) driver. You can create an MC-Hologres external table in MaxCompute to query the data of MC-Hologres data sources by using the JDBC driver that is provided by PostgreSQL. This method prevents redundant data storage and allows you to obtain query results at a fast speed without the need to import or export data. Reading and writing data of MC-Hologres external tables is in the public preview stage. Data computing is free of charge but the Service Level Agreement (SLA) is not guaranteed. | |
2021-02-01 | Change records of Mars V0.6.3 | New description | Change records of Mars V0.6.3 are added. |
Updates in January 2021
Release date | Feature | Category | Description | References |
2021-01-21 | Change in the return behavior of the | Updated description | For new MaxCompute projects, the | |
2021-01-19 | Content adjustment and optimization of the SELECT syntax | Updated description | The content of the SELECT syntax is adjusted, and sample statements are optimized. | |
2021-01-05 | Description of data migration | New description | The Data migration topic is added. The content related to data upload and download and the migration tool is moved under this topic. |