2024 Feature Release Notes - AnalyticDB - Alibaba Cloud Documentation Center

Usage notes

Take note of the following items during minor version updates of AnalyticDB for MySQL clusters:

For AnalyticDB for MySQL clusters in reserved mode for Cluster Edition or AnalyticDB for MySQL clusters in elastic mode for Cluster Edition that have 32 cores or more, data read and write operations are not interrupted when engine versions are updated. Within 5 minutes before the update is complete, queries may encounter transient connections.
For AnalyticDB for MySQL clusters in elastic mode for Cluster Edition that have 8 or 16 cores, data write operations may be interrupted for 30 minutes when engine versions are updated. Within 5 minutes before the update is complete, queries may encounter transient connections.
Minor version updates of AnalyticDB for MySQL clusters do not affect database access, account management, database management, or IP address whitelist settings.
During a minor version update of an AnalyticDB for MySQL cluster, network jitters may occur and affect write and query operations. Make sure that your application is configured to automatically reconnect to the AnalyticDB for MySQL cluster.
During a minor version update of an AnalyticDB for MySQL cluster, the cluster may encounter transient connections. Make sure that your application is configured to automatically reconnect to the AnalyticDB for MySQL cluster.

If you do not need to update the minor version of your AnalyticDB for MySQL cluster or an error occurs during the update process, you can cancel the scheduled minor version update. You can cancel only the scheduled events of a minor version update. For more information, see the "Cancel scheduled events" section of the Manage O&M events topic.

Warning

If the minor version of your AnalyticDB for MySQL cluster is earlier than the latest minor version, Alibaba Cloud pushes a notification at an irregular interval to inform you that the cluster needs to be updated to the latest minor version. We recommend that you update the minor version of your AnalyticDB for MySQL cluster at the earliest opportunity within six months after you receive the notification. Otherwise, you shall assume all liabilities for risks such as service interruptions and data loss.

December 2024

Category	Feature	Description	Documentation
New feature	Cross-account cluster cloning	You can now clone Enterprise Edition, Basic Edition, and Data Lakehouse Edition clusters to another Alibaba Cloud account.	Clone a cluster
	Disk encryption	You can now use the console to check a cluster's disk encryption status and view its key ID.	Disk encryption
	New regions	You can now purchase Enterprise Edition, Basic Edition, and Data Lakehouse Edition clusters in the Thailand (Bangkok) and Philippines (Manila) regions.	Pricing

November 2024

Category	Feature	Description	References
New feature	lake cache	After you enable the LakeCache feature, it caches hot files from OSS on high-performance NVMe SSD storage when you read data from OSS to improve the read efficiency of OSS data.	lake cache

October 2024

Category	Feature	Description	References
New feature	Backup and restoration	You can now delete data backup sets or disable the backup feature in the console.	Manage backups
New feature	Zero-ETL	Zero-ETL supports Lindorm data sources. You can create Lindorm zero-ETL tasks to synchronize and manage Lindorm data from a single location, which integrates transaction processing and data analysis.	Synchronize Lindorm data by using Zero-ETL

September 2024

Category	Feature	Description	References
New feature	Cross-region cluster cloning	You can now clone a source cluster to another region.	Clone a cluster

V3.2.2

Category	Feature	Description	References
New feature	Batch creation of MaxCompute external tables	You can now create multiple MaxCompute external tables at a time.	IMPORT FOREIGN SCHEMA
	New aggregate functions for fast refresh of materialized views	Fast refresh for materialized views now supports the `MAX`, `MIN`, `APPROX_DISTINCT`, `COUNT(DISTINCT)`, and `AVG` aggregate functions in the `QUERY BODY`.	Refresh materialized views
	Access MaxCompute external tables using the Arrow API	You can now use the Arrow API to read data from and write data to MaxCompute external tables. Compared to the traditional Tunnel mode, the Arrow API is more efficient for data access and processing.	Import MaxCompute data using external tables
	INSERT INTO	You can now use a nested TIMESTAMP function in an `INSERT INTO` statement.	INSERT INTO
	Next-generation storage engine	AnalyticDB for MySQL introduces `XUANWU_V2`, a next-generation storage engine. This engine caches cold data on ESSDs to accelerate data reads. Its next-generation column-oriented storage provides higher I/O concurrency and uses less memory. You can also enable the compaction service, which uses a dedicated resource pool to build data remotely. This reduces resource contention on your online services and improves stability.	Introduction to the XUANWU_V2 engine
Optimization	FROM_UNIXTIME function	The FROM_UNIXTIME function now supports formatting an INT-type UNIX timestamp with second precision using DATE_FORMAT.	Date and time functions
Bug fix	Data type conversion	Fixed an issue where converting an integer data type (TINYINT, SMALLINT, INT, or BIGINT) to the DECIMAL type returned an error.	None

August 2024

Category	Feature	Description	References
New feature	Support for Spark engine in interactive resource groups	You can now select the Spark engine when creating an interactive resource group for Data Lakehouse Edition clusters. This dedicates the resource group to processing Spark jobs interactively.	Create and manage a resource group
New feature	Limits on the number of zero-ETL tasks	A limit now applies to the number of zero-ETL tasks that synchronize data from RDS for MySQL and PolarDB for MySQL to AnalyticDB for MySQL.	Use zero-ETL to synchronize data from RDS for MySQL Use zero-ETL to synchronize data from PolarDB for MySQL

July 2024

V3.2.1

Category	Feature	Description	Related documentation
New	Incremental refresh for multi-table materialized views	You can now incrementally refresh multi-table materialized views. This feature incrementally updates the materialized view with data from multiple joined tables, improving query performance and data analysis efficiency.	Refresh a materialized view
	User-defined function (UDF) with the REMOTE_CALL function	You can now use the REMOTE_CALL function to invoke user-defined functions (UDFs) from Function Compute (FC) within AnalyticDB for MySQL.	User-defined functions (UDFs)
	Forcibly delete a database	You can now add the CASCADE keyword to the DROP DATABASE statement to forcibly delete a database and all tables within it.	DROP DATABASE
	Wide table engine	Data Lakehouse Edition clusters provide a wide table engine. This engine is compatible with the open-source columnar database ClickHouse. It can process large volumes of columnar data and is ideal for aggregate queries and analysis on wide-column tables.	wide table engine
	Path analysis functions	You can now use the SEQUENCE_MATCH and SEQUENCE_COUNT functions to analyze user behavior paths and check whether a user's behavior matches an input pattern.	Path analysis functions
	Configure SSL encryption	You can now enable SSL encryption to encrypt data transmitted between a Data Warehouse Edition cluster and its clients. This prevents the data from being monitored, intercepted, or tampered with by third parties.	Configure SSL encryption
	Support for complex data types in MaxCompute external tables	Data Lakehouse Edition clusters now support complex data types, including ARRAY, MAP, and STRUCT, in MaxCompute external tables.	CREATE EXTERNAL TABLE
	Support for the ROARING BITMAP data type in internal tables	The ROARING BITMAP data type is now supported.	User segmentation functions (Roaring Bitmap)
	Flink subscriptions to AnalyticDB for MySQL binary logs	You can now use Flink to consume AnalyticDB for MySQL binary logs in real time.	Subscribe to binary logs by using Flink
	DTS subscriptions to AnalyticDB for MySQL binary logs	You can now use DTS to consume AnalyticDB for MySQL binary logs in real time.	Subscribe to binary logs by using DTS
Improvement	LIFECYCLE keyword is now optional	If you do not specify the LIFECYCLE keyword when you create a table, its partition data is retained permanently.	CREATE TABLE
	Table-level partition lifecycle management (LIFECYCLE)	For kernel versions 3.2.1.1 and later, partition lifecycle management for tables that use the `XUANWU` engine now operates at the table level instead of the shard level. This means that if you specify `LIFECYCLE n`, `n` partitions are retained for the entire table instead of for each shard. For tables that use the `XUANWU_V2` engine, partition lifecycle management remains at the shard level, and table-level management is not yet supported.	CREATE TABLE
	Import OSS data to AnalyticDB for MySQL by using external tables	You can now specify a file name prefix or a wildcard in the Location parameter when you import data from OSS to AnalyticDB for MySQL by using an external table.	Use external tables to import data to a Data Warehouse Edition cluster
	Automatic check for column names during table creation	When creating a table with the CREATE TABLE statement, AnalyticDB for MySQL automatically validates the column names. If a column name is invalid, an error is returned. For more information about the naming rules for columns, see Limits.	None

May 2024

Category	Feature	Description	References
New feature	cross-account cluster cloning	You can clone a Data Lakehouse Edition cluster to another Alibaba Cloud account.	Clone a cluster

April 2024

Category	Feature	Description	Related documentation
New	Query rewrite	The query rewrite feature for materialized views is now supported. After you enable this feature, the optimizer determines whether a query can use precomputed results stored in a materialized view and rewrites part or all of the original query to use the materialized view.	Query rewrite for materialized views
	Synchronize data from SLS	You can now use data synchronization to synchronize data generated in SLS after a specified point in time to your AnalyticDB for MySQL cluster in real time. This allows you to perform real-time analysis on your log data.	Use data synchronization to synchronize data from SLS to a Data Warehouse Edition cluster Use data synchronization to synchronize data from SLS to a Lakehouse Edition cluster
	Zero-ETL data integration	Zero-ETL data integration is now supported. You can create synchronization pipelines from RDS for MySQL and PolarDB for MySQL. It provides an end-to-end solution for data synchronization and management, integrating transaction processing with data analytics.	Zero-ETL data integration
	Select a time zone during cluster creation	You can now select a time zone for your cluster during creation. Once set, all time-related data writes will use the selected time zone.	Create a cluster
	Self-service version upgrade	You can now view and upgrade the kernel version of Data Warehouse Edition clusters from the console.	View and upgrade a version
	Vertical scaling for reserved storage	Lakehouse Edition clusters now support vertical scaling for reserved storage.	Scale a Lakehouse Edition cluster
	Use the Spark Distribution SQL Engine with DataWorks	In DataWorks, you can now configure the Spark Distribution SQL Engine from a Lakehouse Edition cluster as an execution engine by registering a CDH cluster. This lets you develop and run Spark SQL tasks directly in DataWorks.	Use the Spark Distribution SQL Engine with DataWorks
	Progress bar for cluster creation and scaling	When you create or scale a Data Warehouse Edition cluster, the console now displays a progress bar, allowing you to track the task's progress.	Create a cluster

March 2024

Category	Feature	Description	Related documentation
New	Spot instance	Lakehouse edition clusters now support spot instances for job resource groups. Once enabled, Spark jobs on these resource groups will try to use spot instances. Spot instances cost less than elastic ACU resources, helping you reduce the cost of your Spark jobs.	Spot instance

February 2024

Category	Feature	Description	References
New feature	intelligent assistant	The console provides an intelligent assistant. The intelligent assistant answers questions to help you resolve issues quickly. Note The intelligent assistant supports only Chinese Q&A.	None
	Spark Distribution SQL Engine	Data Lakehouse Edition clusters provide a managed service for the open-source Spark Distribution SQL Engine. You can use the Spark Distribution SQL Engine to develop Spark SQL jobs. This simplifies SQL-based data analysis, processing, and querying, and improves execution efficiency.	Use the Spark Distribution SQL Engine to develop Spark SQL jobs
	Access OSS-HDFS data sources	Data Lakehouse Edition now supports accessing OSS-HDFS data sources with Spark.	Access OSS-HDFS data sources
	storage overview	You can quickly view cluster-level and table-level data volumes on the storage overview page.	Storage analysis

V3.1.10

Category	Feature	Description	References
New feature	Primary and foreign key constraint	You can use primary and foreign key constraints to eliminate unnecessary JOIN operations. This reduces query time and improves database performance.	Eliminate redundant JOINs by using primary and foreign key constraints
	Monthly scheduling for resource scaling plans	Data Warehouse Edition now supports setting a monthly cycle for resource scaling plans.	Create a resource scaling plan
	Multi-cluster scaling model	Data Lakehouse Edition resource groups now support the Multi-Cluster feature. After you enable this feature, the Multi-cluster scaling model automatically scales resources based on workloads to provide resource isolation and high concurrency.	Multi-cluster scaling model
	Variable-length binary functions	Added the AES_DECRYPT_MY and AES_ENCRYPT_MY functions.	Variable-length binary functions
	JSON functions	Added the JSON_REMOVE function.	JSON functions
	PlanCache	The PlanCache feature caches the execution plans of SQL statements. When you run SQL statements with the same pattern, the system reuses the cached execution plan. This reduces SQL compilation and optimization time and improves query performance.	PlanCache
	elastic import	Data Lakehouse Edition clusters now support elastic data import. Elastic import can significantly reduce, or even eliminate, storage and compute resource consumption. This feature minimizes the impact of import operations on online read and write services and improves resource isolation.	Elastic import
	Asynchronous scheduling of ETL tasks in DMS	You can use the task orchestration feature of Data Management (DMS) to asynchronously schedule ETL tasks.	None
	Modify workload management rules	You can use WLM commands to modify existing workload management rules.	WLM
Enhancement	basic statistics	Improved the collection strategy for basic statistics.	None
	column group statistics	Improved the collection strategy for column group statistics.	None
	Internal Error messages	Optimized Internal Error messages to help users analyze and locate issues more quickly.	None
	Asynchronous split generation	For large external tables, you can generate splits asynchronously during the execution plan generation phase to reduce processing time.	None
	Concurrency control for data source scanning	Optimized the concurrency control capability (split flow control) for scanning OSS and MaxCompute external tables.	None
	Parameter check policy for RC HTTP API calls	Optimized the parameter check policy for calls made from the console to the RC HTTP component to prevent SQL injection.	None
	Memory usage of storage nodes	Optimized the memory usage of storage nodes to reduce garbage collection (GC) frequency and improve system stability.	None
Fixed issue	materialized view	Fixed an issue where an error was returned for the array_agg function when you created a view by using the CREATE VIEW statement.	None
	Import local data by using LOAD DATA	Fixed issues with CSV file incompatibility and incorrect data segmentation that occurred when you imported local data to a Data Warehouse Edition cluster by using the LOAD DATA statement.	None
	cold data storage	Fixed an issue with cold data storage, resulting in an improved query hit rate and better query performance.	None