PolarDB:PolarDB for MySQL 8.0.1 release notes

Description

• Added support for sharded table scenarios in Outline. A single Outline can match all statements with the same template across different sharded tables.

• Added the ability to dynamically adjust the Order Index selection during execution based on runtime conditions.

• Introduced parallel Redo log parsing for physical replication to improve physical replication performance.

• Added the ignore_use_force_index variable to ignore all use or force index hints in statements and Outlines when enabled (this does not affect Index() in a hint).

• Added the convert_sys_to_filename() and convert_filename_to_sys() functions to convert strings between the system_charset_info and my_charset_filename encodings.

• Added support for transparent data encryption (TDE) for cold data in OSS.

• Added support for asymmetric MTT buffers on read/write (RW) and read-only (RO) nodes to enable coordinated MTT optimization between them.

• The IMCI i_s.imci_index_stats table includes a new PRUNED_PERCENT field, which indicates the Data Skipping percentage for each column index.

• Enhanced SHOW ProcessList to display whether a session is waiting for thread pool scheduling.

• You can directly rebuild a columnstore index by modifying pack_shift in the COMMENT without first deleting the index.

Description

• Optimized the dynamic Buffer Pool shrink process to reduce lock contention.

• Optimized SQL Trace to correctly record SQL and the execution plan when you use Call procedure.

• Reduced the memory overhead of wide tables with columnstore indexes.

• Improved the startup speed of columnstore read-only nodes. The columnstore engine can now start independently from a columnstore snapshot LSN.

• Optimized the thread creation and termination code for logical prefetching and simplified the memory check logic.

• Reduced the startup time for column store nodes in scenarios with a large number of tables.

• Improved the performance and reduced the memory usage for large transactions in IMCI.

• Optimized the performance of materialized hash key expressions in IMCI.

• Optimized the serverless process for column store nodes. New column store nodes are now added to the proxy to handle traffic only after their indexes are ready.

• Improved the statistics sampling mechanism in IMCI by increasing the sampling frequency and the proportion of sampled data.

  Optimized the sampling logic for low-cardinality columns to prevent suboptimal query plans caused by inaccurate statistics.

• Optimized the Simulated-AIO asynchronous performance blocking module.

Description

• Fixed an issue where DDL statements on tables with views would fail or not take effect after an HA (node switchover).

• Fixed an issue where a read-only node might remain out of service after reconnecting to a read/write node.

• Fixed an issue where the CAST(REGEXP_SUBSTR(...)) AS DECIMAL statement caused a process to crash.

• Fixed an issue where REGEXP_SUBSTR might incorrectly return NULL.

• Fixed an issue where the CREATE TABLE statement for the dlm_policies system table was recorded in the BINLOG, causing downstream synchronization to fail.

• Fixed an issue where unique indexes were not fully utilized to accelerate some queries.

• Fixed an issue where using the ALTER TABLE DROP statement to delete multiple full-text indexes (FTS) at the same time could leave residual metadata.

• Fixed unnecessary index rebuilds when extending the length of a VARCHAR column with a specified DEFAULT value.

• Fixed multiple issues with IMCI serverless, including compatibility with internal memory elasticity and abnormal scaling behavior.

• Fixed an issue where full-text index retrieval would get stuck on a read-only node after a DDL operation.

• Fixed an issue in adaptive execution where certain types of queries were not routed correctly.

• Fixed an IMCI availability issue caused by a lack of support for online rebuilds.

• Fixed a crash that occurred when batch adding columnstore indexes in a database- or table-level multi-master cluster due to incomplete adaptation to the DDL global lock.

• Fixed a crash that occurred in queries on JSON_TABLE in scenarios involving a NULL field.

• Fixed a conflict between multi-master Buffer Pool resizing and transaction block access, and optimized the performance of RDMA register or unregister operations for the multi-master Buffer Pool.

• Fixed an issue where Digest CCL did not support the Prepared Statements (PS) protocol.

• Fixed an issue where a read-only node could crash during an HA phase after the RO No Sync feature was enabled.

• Fixed an issue where transaction rollbacks for spatial indexes required a pessimistic lock to delete the index, which could lead to an infinite loop and prolonged lock holding.

• Fixed a rare issue where an upgrade task from a previous version to 8.0.1.1.45.1 could be interrupted.

• Fixed a rare issue where a query error occurred due to accessing invalid data when processing string data types during column store execution.

• Fixed an issue where IMCI would fall back to the row store when processing an expression that contained multiple subqueries with SUM.

• Fixed a circular wait issue during asynchronous column index rebuilding in a one-writer, multiple-reader IMCI setup when using Hybrid Plan to accelerate wide table queries was enabled.

• Fixed an issue where modifying the innodb_bulk_load_page_grained_redo_enable parameter within a specific time window during parallel DDL execution caused the Redo of some pages to be written incorrectly.

• Fixed a potential deadlock in BLINK during batch data deletion from tables containing LOB fields.

• Fixed a crash during the migrate process caused by a concurrency issue between high-frequency INSTANT DDL operations and background compaction tasks.

• Fixed an issue where GROUP BY could cause a temporary table to report a table is full error.

• Fixed an issue where an index join in IMCI could produce unstable results when a constant was used as a join condition.

Description

• Fixed an issue where executing an INSERT ... ON DUPLICATE KEY UPDATE statement after a switchover from an RO node to an RW node caused the auto-increment value of a table to roll back.

• Fixed multiple issues with IMCI serverless, including compatibility with internal memory elasticity and abnormal scaling behavior.

• Fixed a crash that occurred when the number of tables in a query transformation exceeded 61. Query rewrite is now skipped in this scenario.

Description

• Fixed an issue where full-text index retrieval would get stuck on a read-only node after a DDL operation.

• Fixed an incompatibility between the backward index scan feature and optimizations such as ICP, and added the polar_optimizer_switch parameter to control this feature.

Description

Added the ignore_use_force_index variable, which can be enabled to ignore all use/force index in statements and Outlines (but does not affect the index() hint).

Description

• Added the PRUNED_PERCENT field to the INFORMATION_SCHEMA.IMCI_INDEX_STATS table in IMCI. This field indicates the data skipping ratio for each columnstore index.

• Optimized SQL Trace to also apply to statements specified by DEMAND when the trace type is set to SLOW LOG.

• Optimized support for asymmetric MTT buffers on read/write and read-only nodes, allowing for coordinated enabling of MTT between them.

• Optimized the serverless process for column store nodes.

  New column store nodes are allowed to serve traffic only after their indexes are ready. This reduces slow queries caused by SQL statements being routed to new nodes and falling back to row store execution plans.

• Reduced the performance overhead of saving partition information during SQL execution in IMCI to improve query performance on partitioned tables.

Description

• Fixed a crash that occurred during the Undo initialization phase when Force recovery was enabled.

• Fixed a circular wait issue during asynchronous column index rebuilding in a one-writer, multiple-reader IMCI setup when Hybrid Plan was enabled.

• Fixed an error that occurred when re-importing the same data into a UNIQUE CHECK IGNORE partitioned table by using LOAD DATA with REPLACE.

• Fixed an issue where recent_written could introduce dirty data in a three-availability-zone deployment when the CRC value of the input data is exactly 0.

• Fixed an issue where a read-only node would get stuck during an HA (node switchover), causing prolonged HA times.

• Fixed B-tree corruption that occurred when adding an auto-increment column and creating a descending primary key index on it.

• Fixed an issue where IMCI would fall back to the row store when processing an expression that contained multiple subqueries with SUM.

• Fixed a rare memory leak that occurred when an error was encountered during parallel primary key rebuilding.

• Fixed a potential crash that occurred when using the Temptable engine for GROUP BY queries that process LOB fields.

• Fixed an issue where parallel query results could be incorrect if the predicate condition contained a subquery.

• Fixed a potential query crash in JSON_TABLE scenarios involving a NULL field.

• Fixed an NCI encoding anomaly that occurred during EQUAL_PACK_LENGTH DDL operations when modifying a PRIMARY KEY field.

• Fixed a conflict between advanced transaction splitting in the proxy and Fast Query Cache that affected the result set.

• Fixed an issue where using a JSON_ARRAY expression in a subquery could lead to incorrect results when IMCI was enabled.

• Fixed a process interruption in the new workflow for database and table recovery that occurred after multiple recovery rounds on the same table.

• Fixed a memory leak that occurred during the dynamic enabling of the RDMA log transfer feature.

• Fixed an availability issue where an IMCI could not be rebuilt online.

• Fixed an issue where asynchronous DDL could not be used when adding an IMCI by modifying the COMMENT on a table with a full-text index.

• Fixed an issue where the kill operation on a transaction might fail in high-concurrency scenarios.

Description

• Fixed a rare issue where an upgrade task from a previous version to 8.0.1.1.45.1 could be interrupted.

• Fixed a rare query error caused by accessing invalid data when processing string data types during column store execution.

Description

Reduced the memory overhead of IMCI for wide tables.

Description

• Added a status indicator to show whether a session is waiting for thread pool scheduling.

• Introduced a greedy algorithm-based join reorder strategy in IMCI.

• Introduced a stored procedure and an extended SQL command to batch-add columnstore indexes to an entire database.

• Introduced a feature for major version upgrades from a PolarDB 5.7 backup set to improve one-click migration efficiency.

• Expanded IMCI Hybrid Plan support to include partitioned tables, virtual columns, and columns with ENUM, SET, and GIS data types.

Description

• Optimized PolarDB I/O threads to be dynamically adjustable without requiring a restart.

• Improved the auto-scaling capability for single-node replay in IMCI to reduce replay latency and prevent out-of-memory (OOM) issues under high replay pressure.

• Optimizes the read performance of Date-type fields in IMCI.

• Improved the statistics sampling mechanism in IMCI by increasing the sampling frequency and the proportion of sampled data.

  Optimized the sampling logic for low-cardinality columns to prevent suboptimal query plans caused by inaccurate statistics.

Description

• Fixed a node crash caused by InnoDB full-text indexes in a multi-master cluster.

• Fixed an issue where the optimizer significantly misestimated selectivity when a JOIN condition in IMCI included a type conversion.

  Fixed an issue where a Hybrid Plan was not correctly selected in IMCI due to an overestimation of the output cardinality of a JOIN.

• Fixed an issue where JSON_LENGTH returned incorrect results when wildcards were used.

• Fixed an issue where modifying the innodb_bulk_load_page_grained_redo_enable parameter within a specific time window during parallel DDL execution caused the Redo logs for some pages to be written incorrectly.

• Fixed an issue where the AUTO_INCREMENT value of a partitioned table would roll back after a read-only node was switched to a read/write node.

• Fixed an issue where removing an RO node could cause the monitoring connection to be blocked for a period of time when innodb_polar_log_rdma_transfer is enabled.

• Fixed an issue where an IMCI index join could trigger unstable results when a constant is used as the JOIN condition.

• Fixed an issue where using a group by statement on a temporary table could cause a table is full error even when the disk was not full.

• Fixed an issue where user threads would wait excessively for a free page on a read-only node when the buffer pool was nearly full.

• Fixed an issue that could cause a deadlock in Blink when you batch delete data from a table containing a lob field.

• Fixed an issue where using JOIN elimination optimization in IMCI could lead to unstable results.

• Fixed an issue where the AUTO_INCREMENT value would restart from 1 on the next write operation after all data in a table had been deleted.

• Fixed an issue where the drop database command fails on a database that contains tens of thousands of tables.

• Fixed an issue where unneeded subqueries were not removed during view merging.

• Fixed an issue where a query with too many UNION ALL clauses could report an out-of-memory error when executed in IMCI.

• Fixed an issue where desc select statements could not correctly call outlines.

• Fixed an issue where in information_schema.tables, the table_comments column displays xxx is not base table.

• Fixed an issue where implicit conversion of date types in IMCI was incompatible with MySQL behavior, leading to unexpected results.

• Fixed an issue where an inconsistent default pruner type caused inconsistent query results when a new partition was added to an IMCI partitioned table that was created in a previous version after an upgrade.

• Fixed a floating-point precision issue in JSON.

Description

• Supports deriving the corresponding view from a join condition.

• Introduced a late-materialization version of UnionAll for IMCI.

Description

• Reduced query optimization time for simple (single-table) queries.

• Added support for fine-grained table-level locks in multi-master clusters.

• Improved the performance of decimal sum/avg in IMCI.

• Improved performance in hot-spot update scenarios.

• Improved the performance of hash match in IMCI.

• Reduced memory usage for metadata in IMCI queries.

• Added support for filter pushdown to outer join in IMCI.

• Adjust the PolarFS transaction processing thread priority to improve PolarDB DDL performance.

Description

• Fixed an issue where a hung Blink page caused the import table command to fail.

• Fixed an issue where the database would hang during a manually triggered checkpoint in various processes.

• Fixed an issue where markers could be lost if RO No Sync was toggled frequently.

• Fixed an issue where a global standby node in a multi-master cluster could not start due to an abnormal rollback segment.

• Fixed an issue where InnoDB tables with a function index could not be archived.

• Fixed an issue where Last_query_cost returned a negative value in some optimization scenarios.

• Fixed an issue where join elimination could not be performed if a function contained an aggregate function.

• Fixed an issue where the IP address in the audit log was not updated in a timely manner when connections were reused in a session-level connection pool.

• Fixed issues related to acquiring and releasing global table locks in a multi-master cluster.

Description

• Added binary log support for resumable transactions.

• Added a new parameter thread_pool_reset_waiting_state_before_queuing.

Description

• The columnstore index (IMCI) adds support for Index Join based on a Hybrid Plan, which enables you to use InnoDB indexes to perform a Join.

• Improved the page read performance of read-only nodes on Polar File System (PFS).

• Reduced memory usage for metadata in IMCI queries.

Description

• Fixed an issue where setting certain invalid values for optimizer_switch and parallel_query_switch would report an Error instead of a Warning.

• Fixed an issue where the optimizer_switch value did not take effect if it ended with a comma.

• Fixed an issue where the index selection changed for implicit aggregate queries with a limit operation.

• Fixed an issue where the undo log grew continuously after a follower was rebuilt in a three-availability-zone deployment.

• Fixed an issue where Persist RO could cause a read-only node to read stale data.

• Fixed an issue of high replication delay on read-only nodes in a multi-master cluster.

• Fixed an issue where the optimizer could not select the optimal Join Order due to an error in cardinality estimation.

• Fixed a deadlock between threads that could occur during B-tree restructuring caused by record deletion.

• Fixed an issue where archiving a table with a descending index failed.

• Fixed an issue where LIMIT OFFSET pushdown on a query with an equality condition on a composite secondary index and a primary key In List condition returned no data when paginating results in reverse primary key order.

• Fixed excessive permissions during Data Lifecycle Management (DLM) policy execution.

• Fixed an issue where executing a Create Table Select statement in a multi-master cluster affected the availability of other nodes.

• Fixed an issue where system pages could be corrupted if DDL statements were run on multiple nodes at the same time in a multi-master cluster.

Description

• Added predicate pushdown, including pushing predicates from HAVING to WHERE and from HAVING to derived tables and subqueries.

• Introduced a hybrid row-column optimizer and a cost-based routing capability for row and column stores.

• Added support for viewing Statement Outline invocations in the EXPLAIN output.

• Added support for using a prefix MurmurHash bloom filter in IMCI.

• Added support for LEFT JOIN elimination in UPDATE statements.

• Added support for instantly adding columns to tables that have an IMCI.

• Enhanced the Fast Query Cache to include the number of returned rows. This count is now shown in the audit log for queries that hit the cache.

Description

• Introduced a feedback mechanism to row-column routing to collect actual execution information and provide it to the optimizer.

• Disabled index merge intersection for DML statements by default to improve performance and prevent deadlocks.

• Reduced the metadata overhead for memory allocation and statistics in IMCI to save memory.

• Added support for BatchProbe in IMCI to improve JOIN performance.

• Optimized memory usage for metadata, file modules, and small resident objects in IMCI to reduce resident memory consumption.

• Improved the error message for importing cold data from OSS back into InnoDB.

• When monitoring database performance, ignore the impact of access from system administrators on the slow_queries status metric.

Description

• Fixed an issue where the OSS path mapping for tables in the recycle bin was incorrect after a cluster restart. This occurred when recycling OSS IBD tables with the RECYCLE BIN feature enabled.

• Fixed an issue where a read-only node with excessive clock drift could preempt the read/write node's permissions when starting the NewHA vector voting disk.

• Fixed an issue where the dbms_oss.delete_table_file built-in stored procedure could not delete cold data in CSV format.

• Fixed an issue where the internal counter of the thread pool could become incorrect if an ACL lock wait was encountered during the asynchronous authentication process.

• Fixed an issue where space for original Blob data was not reclaimed after an UPDATE operation on a Blob column.

• Fixed a data loss issue that occurred when repeatedly archiving an OSS external table.

• Fixed an issue where the sorting operation was not omitted at runtime even though the optimizer had selected an ordering index.

• Fixed a node crash that occurred when creating an OSS external table because the oss_prefix parameter was not configured on the OSS server.

• Fixed an issue where abnormal data could be retrieved when accessing some tables in the information_schema database in a multi-master cluster.

• Fixed an issue where an assert message was returned when multiplying Decimal data if at least one multiplier was 0 and the result's scale exceeded 16.

• Fixed an issue where updating statistics on a partitioned table with an IMCI could result in inaccurate statistics because only a subset of partitions was sampled.

• Fixed an issue where index equality access did not trigger adaptive row-column routing.

• Fixed an issue of excessive Autoinc init log printing.

• Fixed a potential error in Order By Desc scenarios for OSS external tables.

• Fixed a rare node crash that occurred when executing the TRUNCATE PARTITION command on a partitioned table with an auto-increment column.

• Fixed an issue where data corruption occurred due to different index orders when you perform an EXCHANGE PARTITION operation.

• Fixed a potential node crash under high concurrency, caused by the Buffer Pool's automatic cancellation of a preceding shrink operation.

Description

Added a Left Join feature based on TopK pushdown in IMCI.

Description

• You can grant users the global dynamic privileges to execute the SHOW_ROUTINE, FLUSH_USER_RESOURCES, FLUSH_TABLES, FLUSH_STATUS, and FLUSH_OPTIMIZER_COSTS commands.

• The process for enabling the high-performance mode for global consistency has been optimized. When the innodb_polar_scc parameter is set to ON on an RO node, the cluster automatically enables dependent features, such as RDMA and PolarTrans, which simplifies the management process.

• Added support for using the Variable-Setting Hint syntax in high-performance global consistency mode.

• You can set the polar_oss_ddl_shared parameter to ON to access data in a table that is undergoing cold data archiving.

• Disabled the partial update feature for Blob fields.

• Added support for using the database and table recovery feature for tables with an IMCI.

• Added support for most common value (MCV) statistics collection in IMCI to improve the accuracy of records-per-key estimation in data skew scenarios.

• Optimized the memory scheduling feature for IMCI to improve memory utilization.

• Enhanced the non-blocking DDL feature to support more DDL statements.

• Added support for using IMCI and global consistency (high-performance mode) features at the same time.

• Added support for using the CHECK TABLE command to check a specific partition.

Description

• Fixed an issue where a TRUNCATE statement could be run on any node in a multi-master cluster.

• Fixed a node crash caused by accessing an invalid Blob reference when pre-reading Blob data.

• Fixed a potential failure when exporting a local table to the OSS engine in parallel.

• Fixed an issue where the crash recovery logic could not run normally after an abnormal cluster restart. This was caused by an unexpected checkpoint LSN during a checkpoint operation.

• Fixed an issue where an AUTO_INCREMENT rollback required a node restart to recover.

• Fixed a potential node crash when a Common Table Expression (CTE) referenced a TempTable-based temporary table multiple times and used an index scan on it.

• Fixed a memory out-of-bounds issue when IMCI processed extra-long string types.

• Fixed a random node crash that occurred when the pre-read feature was enabled while IMCI was in use.

• Fixed an issue where the X-Engine compaction background task negatively affected foreground operations and slow query performance due to overly strict concurrency control.

• Fixed an issue where using a BIGINT column in a Join operation could lead to inaccurate cardinality estimation and prevent the use of an IMCI if the column contained very large values.

• Fixed a deadlock in DDL pre-read in parallel DDL scenarios.

• Fixed a memory leak that occurred during Reload operations for features like Statement Outline and Concurrency Control.

• Fixed an issue where BIT fields were displayed incorrectly after a Group By operation.

• Fixed an incompatibility between TDE and RDMA memory when forking a child process.

• Fixed an issue where duplicate groups appeared in the result set when a parallel query pushed GROUP BY down to a worker for parallel execution on a partitioned table.

• Fixed an issue where running an EXCHANGE PARTITION operation could cause index corruption and prevent access to table data due to different index orders.

• Fixed issues related to partitioned tables whose primary key did not include the partition key during query optimization, DML operations, and binary log recording.

Description

• Fixed an issue where the replication delay of a global read-only node could increase if the load on the primary nodes of a multi-master cluster was unbalanced.

• Fixed an issue where a newly created read-only node might fail to start on a multi-master cluster.

• Fixed an issue where running a DDL statement on a multi-master cluster could cause the primary node to fail to restart.

Description

• Added support for sampling statistics in the X-Engine to estimate the approximate compression ratio.

• Added support for enabling IMCI on a global read-only node of a database- or table-level multi-master cluster.

• Allowed users to ignore transaction isolation levels that are implicitly set by tools like Metabase when using IMCI.

• Added support for Stats Manager to automatically dispatch histogram collection tasks to read-only or hot standby nodes for automatic updates.

• Added support for using temporary tables when using a Hybrid Plan to accelerate wide table queries.

• Introduced a feature to enable adaptive degradation and automatic recovery for semi-synchronous replication.

• Added support for modifying the character set of a field by using the INPLACE algorithm.

• SQL Trace supports enabling configuration options for specified read-only nodes by allowing you to set the sql_trace_type parameter to REPLICA_DEMAND, REPLICA_ALL, or REPLICA_SLOW_QUERY.

Description

• Improved the speed of handling connection requests by the thread pool in low-concurrency scenarios.

• Improved database performance in short-lived connection scenarios.

• Optimized the transaction mask calculation process in IMCI.

• Automatically increased the maximum switchable execution time when adaptive execution encounters a high volume of cold data read I/O.

• Adjusted the maximum display length of query statements in SQL Trace.

• Introduced dynamic filters to the IMCI feature to improve join efficiency between tables.

• When you use SET_VAR in a HINT statement to set the value of the loose_optimizer_switch parameter, do not add a comma at the end.

Description

• Fixed an incorrect sorting result when querying TopK data with IMCI. This occurred when sorting in descending order on a table with a large number of NULL values.

• Fixed an issue where a read-only column store node could not recover column store data if it was started during a DDL operation on a table with an IMCI.

• Fixed a space leak caused by an exception during a shrink operation.

• Fixed an issue where concurrently modifying data structures during physical replication in the X-Engine caused a read-only node to become abnormal.

• Fixed a failure when upgrading an X-Engine node from a read replica to a synchronous standby.

• Fixed excessive optimizer memory usage when using IMCI in high-concurrency query scenarios.

• Fixed an issue where updating statistics on a partitioned table with an IMCI could result in inaccurate statistics because only a subset of partitions was sampled.

• Fixed an issue where a query containing a subquery that returned an empty result set could not be routed to a column store node for execution.

• Fixed an issue where a CCL rule matched by a DIGEST value could become invalid after a cluster upgrade.

• Fixed an issue where querying the information_schema.tables table was abnormally slow when the database contained CSV or ORC external tables.

• Fixes an issue where Flink CDC parsing fails due to a duplicate table_id in the global Binlog of a Multi-master Cluster (Database/Table).

• Fixed an issue where resumable transaction data could not be queried if the parallel query feature was enabled.

• Fixed a failure to add a new read-only node when using the new database and table recovery workflow. This was caused by the read-only node registering with an LSN that was too early.

• Fixed an issue where a thread would not exit due to a full disk when using the new database and table recovery workflow.

• Fixed high memory consumption when using prepared statements to handle many concurrent requests or complex queries.

• Fixed an issue where querying a view containing BIT data with IMCI produced incorrect results.

• Fixed an issue where partition pruning on a KEY partitioned table did not consider generated columns (STORED GENERATED COLUMN), causing INSERT operations to fail with the error "ERROR 1748 (HY000) Found a row not matching the given partition set".

• Fixed an issue where new connections would still access a discarded old file due to frequent DDL and HA operations when archiving cold data to the IBD format.

Category

Description

Release date

New features and performance optimizations

• You can now access the INFORMATION_SCHEMA views of the column store from any node. By default, information from all column store nodes is aggregated and displayed.

• Simplified the parameters for automatic request distribution between the row and column stores.

• If a query does not support the columnstore engine, you can set the use_imci_engine parameter to FORCED and re-execute the query. The query will then return the reason for the incompatibility.

• Supports using and eliminating complex correlated subqueries in the GROUP BY clause.

• The global Binlog of a multi-master cluster now supports the global transaction identifier (GTID).

• Optimized the Buffer Pool expansion feature to automatically cancel previous scale-in operations and ensure timely expansion.

• Optimized the startup process for the columnstore index (IMCI) to reduce its startup time.

• Added support for adaptive routing for hybrid row-column queries.

• Improved read performance on read-only nodes for queries involving primary key lookups via a secondary index.

2023-12-14

Bug fixes

• Fixed a node crash caused by an invalid format of the user-defined CONNECTION string when creating an OSS foreign table.

• Applied a patch from the official MySQL 8.0.17 to fix an auto-increment rollback issue caused by modifying auto_increment_increment on the client.

• Applied a patch from the official MySQL 8.0.16 to fix a deadlock issue caused by an oversized BLOB.

• Fixed a read-only node crash caused by Runtime Apply.

• Fixed blocked dirty page flushing on an RW node. This issue occurred when the RW node failed to sever its replication link with an RO node on a failed physical machine.

• Fixed an auto-increment rollback issue caused by table cache eviction due to an excessive number of tables after a High Availability (HA) event.

• Fixed an issue where a read/write node in a multi-master cluster could fail to start if the cache center was restarted.

• Fixed an issue where a table might fail to open when switching the write point in a multi-master cluster.

• Fixed excessive memory usage during physical replication on a read-only node.

• Fixed an issue where an outline on an AP node did not take effect if the length of the SQL token exceeded the value of the max_digest_length parameter.

• Fixed an issue where added or deleted outlines were not synchronized to a read-only node after the node was upgraded.

• Fixed high response times for low-concurrency writes to tables in X-Engine.

• Fixed an issue where applying a Hybrid Plan to an SQL statement with an ORDER BY clause and NULL values in the output rows would report a "table definition has changed" error.

Category

Description

Release date

Bug fixes

Fixed an issue where character sets affected matching for new Statement Outlines on clusters that were upgraded without existing outlines.

2023-11-07

Category

Description

Release date

New features and performance optimizations

• You can now use the built-in stored procedure dbms_oss.delete_table_file to delete unused files on OSS.

• Optimized the database and table restoration feature, improving restoration speed and supporting task failure rollbacks.

2023-10-31

Bug fixes

• Fixed an issue that prevented point-in-time restores of tables with a columnstore index (IMCI).

• Fixed an issue where the GroupJoin operator consumed excessive memory in some scenarios, causing queries to fail.

• Fixed a rollback failure for advanced RowIDs when loading a checkpoint.

• Fixed a space reclamation failure on X-Engine caused by a read-only (RO) node's inability to report version reference information.

• Fixed an issue where the optimizer inaccurately estimated join selectivity when joining an unsigned column with a signed column.

• Fixed an execution failure for some queries with OR predicates caused by the execution plan failing to bind to the correct columns.

• Fixed an issue where a query terminated by the scheduler could cause a database restart.

• Fixed an I/O performance degradation issue for the columnstore index in late materialization scenarios.

• Fixed a rare issue where a multi-master cluster would crash because it failed to release the metadata lock for a scheduled task in a timely manner.

• Fixed an issue where queries on tables in X-Engine were slow because an index covering scan could not be used after accessing the primary key index.

• Fixed an issue where the migration link was interrupted by a PolarDB cluster restart during a one-click upgrade from RDS for MySQL to PolarDB for MySQL.

• Fixed a replica node crash caused by a race condition between deleting a table or index in X-Engine and a concurrent background SwitchMemtable operation.

Category

Description

Release date

New features and performance optimizations

• Optimized Redo log read performance on read-only (RO) nodes in large BLOB scenarios.

• You can now call built-in stored procedures by using uppercase names.

2023-09-21

Bug fixes

• Fixed a node crash caused by adding backticks to the input parameters when calling some built-in stored procedures.

• Fixed a node crash that could occur when deleting the last CCL rule of a specific SQL statement type where only the keyword was specified without a schema and table.

• Fixed a rare crash during the upgrade of a read-only (RO) node to a read/write (RW) node. The crash occurred if a new RO node connected to the RW node, and the RW node had file extension logs created after the last checkpoint.

Category

Description

Release date

New features and performance optimizations

• Improved the speed of archiving cold data to OSS in CSV format.

• Optimized the cost model for equality non-covering index access.

• Added the opt_non_covering_key_cost parameter to control whether to consider the initial positioning cost of a secondary index.

• Added the worst_seek_records_threshold parameter to control whether to limit the maximum I/O cost of equality non-covering index access during cost estimation.

• Added a timeout degradation mechanism to the Redo log semi-sync feature.

2023-08-22

Bug fixes

• Fixed a crash in the Join Push Down (JPPD) optimization when a Common Table Expression (CTE) contained nested CTEs and had multiple references.

• Fixed an issue where a replica node did not trigger the reconnection mechanism when the source node was shut down and restarted.

• Fixed a potential issue when using the JSON_REMOVE function in an UPDATE statement for BLOB data.

• Fixed an abnormal node restart caused by invalid memory access during transaction deadlock checks.

Category

Description

Release date

Bug fixes

Fixed high CPU usage on low-specification clusters.

2023-07-13

Category

Description

Release date

New features and performance optimizations

• Added the SELECT INTO DEFAULT OSSOUTFILE statement.

• Added the innodb_polar_import_tablespace_strict_mode parameter. When set to ON (default), it prevents database and table restoration for tables that have a full-text index, reporting an error instead.

• You can now use a columnstore index to query data from a specified partition in a partitioned table.

• SQL Trace now supports sampling specific SQL statements.

2023-07-05

Bug fixes

• Fixed an issue where a regular user could drop the sys schema.

• Fixed an abnormal node restart when performing database and table restoration on a partitioned table after a High Availability (HA) event.

• Fixed an issue where a standby node failed to be promoted to the master node during a point-in-time restore of a multi-master cluster.

• Fixed an issue where stored procedures, user-defined functions, scheduled tasks, views, and triggers were not isolated between different nodes in a multi-master cluster.

• Fixed an issue where stored procedures, user-defined functions, scheduled tasks, views, and triggers could not be switched between different nodes in a multi-master cluster.

• Fixed an issue where data in the Information_schema database was not updated synchronously after a data update in a multi-master cluster with the query cache feature enabled.

• Fixed slow cleanup in a multi-master cluster with an excessive number of data tables.

Category

Description

Release date

New features and performance optimizations

• You can now create a columnstore index on a virtual column.

• You can now use the recycle bin for tables that have a columnstore index.

• You can now create a hybrid partition table.

• The federated query engine now supports pushing down the LIMIT ... OFFSET syntax to a remote database for execution.

• The federated query engine now supports pushing down all compatible conditions to a remote server for execution and returns only the columns required by the query.

• Added the primary key data slicing function call dbms_dts.show_parallel_ranges (<schema_name>, <table_name>, [<suggestion_ranges>]).

• You can now use the cold data archiving feature for tables with an auto-increment primary key.

• Optimized OSS query capability. After the elastic parallel query feature is enabled, the system now performs a parallel query directly if the value of the loose_csv_max_oss_threads parameter is greater than 1.

• Improved the error message for an incorrect OSS foreign table file format.

• You can now add the without validation option to the ALTER TABLE PARTITION BY RANGE syntax to quickly convert a standard table to a RANGE-partitioned table.

• You can now convert a partitioned table with the 'UNIQUE CHECK IGNORE=1' attribute to a standard table by using the ALTER TABLE REMOVE PARTITIONING syntax.

• Optimized Statement Outline to skip digest hash calculation for statements that do not have a matching outline.

• Added the InnoDB on OSS feature.

• After Statement Outline and Concurrency Control are updated on a multi-master cluster, the changes are now automatically synchronized to other master nodes.

• Improved error messages for spelling errors.

• Optimized the write performance of the read/write node after Standby semi-sync is enabled.

• Added the LIST DEFAULT HASH partition table type.

• You can now create a LIST DEFAULT HASH partitioned table by using the CREATE LIKE syntax.

• The ADD PARTITION syntax for creating a LIST DEFAULT HASH partitioned table now supports the without validation option to add a LIST partition.

• SQL Trace now supports tracking the execution information of slow queries.

• Optimized the internal lock granularity of PolarFS to significantly improve performance in some high-load scenarios.

2023-06-12

Bug fixes

• Fixed an issue that prevented OSS from reading JSON files.

• Fixed a Hint Plan generation failure in prepared statement (PS) mode.

• Fixed an issue where the checkpoint LSN did not advance as expected.

• Fixed an issue where view_definition was empty in system views.

• Fixed an issue where a query on a read-only node could report an error if an ALTER TABLE ... IMPORT TABLESPACE operation was concurrently running on the read/write node.

• Fixed an issue that prevented reading data containing \n after it was archived by Data Lifecycle Management (DLM).

• Fixed an issue where temporary files might remain on OSS after cold data archiving.

• Fixed an issue where only the first file might be read when reading CSV-formatted cold data from OSS.

• Fixed an issue where an abnormal internal counter in the thread pool could cause SQL requests to pile up during concurrent logging of slow queries.

• Fixed a potential auto-increment column conflict during concurrent INSERT command execution in a multi-master cluster.

• Fixed a replication lag issue when replaying Redo logs of a wide table on a read-only node.

• Fixed an incorrect query result caused by the Limit Offset pushdown optimization when used with a query containing a window function.

• Fixed an issue where a lock was held for the entire duration of a CHECK TABLE statement execution in a multi-master cluster.

• Fixed an issue where the source file on OSS could not be deleted when executing the ALTER TABLE engine = csv storage oss, force; command.

• Fixed a cluster crash caused by dynamically adjusting the cache of a multi-master cluster.

• Fixed an issue where a multi-master cluster could not export a tablespace.

• Fixed an incorrect pushdown for tables with NULL fields when the engine_condition_pushdown parameter was set to true.

• Fixed a potential incorrect result set issue when comparing Time and Timestamp values with the Fast Query Cache feature.

• Fixed a potential performance regression in high-concurrency update scenarios after PolarTrans was dynamically enabled.

Category

Description

Release date

New features and performance optimizations

• The columnstore index now supports instant column addition by rebuilding columnstore index data.

• The columnstore index now supports the REGEXPR function.

• The columnstore index now supports pruning optimization based on IS NULL and IS NOT NULL conditions.

• Multi-master Cluster (Database/Table) now supports the cold data archiving feature.

• Supports parallel upgrades of nodes within a Multi-master Cluster (Database/Table) during a minor engine version update.

• Multi-master Cluster (Database/Table) now supports persisting statistics.

• Optimized the optimizer trace when OSS condition pushdown is enabled for an OSS foreign table.

• Manual and automatic cold data archiving no longer records Binlog.

• For Global Consistency (High-Performance Mode), added the Innodb_scc_timeout_count parameter to record the number of query timeouts.

• For Global Consistency (High-Performance Mode), added the Innodb_scc_timeout_degrade_count parameter to record the number of times a query times out and degrades to an asynchronous query.

• You can now query GTID information on read-only nodes.

• Hint Plan now supports the BNL, BKA, and MRR hint types.

• Fast Query Cache now includes the qc_weak_read_time variable for reading non-real-time cache.

• The Statement Outline feature now includes the dbms_outln.add_optimizer_outline('schema', 'hints', 'query') interface for adding Optimizer Hints.

• Added the Auto Plan Cache feature.

• SQL Sharing now displays plan_extra, ref_by, and extra in JSON format.

• The slow query log's extended data now includes the execution plan.

• When using Statement Outline, the system now reports a warning instead of an error if the specified Index Hints do not exist.

• SQL Sharing now includes the by_expr_ignore_charset variable. When by_expr_ignore_charset is set to ON, digest differences generated by statements containing ORDER BY or GROUP BY under different character sets are eliminated. However, after this variable is set to ON, all digest values generated in this mode will differ from those generated before it was set to ON.

• Optimized memory consumption when scanning tables in the recycle bin.

• Introduced a multi-version engine.

2023-04-14

Bug fixes

• Fixed a node crash that occurred when running a CHECK operation on an empty OSS foreign table.

• Fixed an occasional query error on a global read-only node of a Multi-master Cluster (Database/Table).

• Fixed an issue where the metadata tablespace size was not synchronized in a Multi-master Cluster (Database/Table).

• Fixed an issue where data in custom tables within the system tablespace was rolled back in a Multi-master Cluster (Database/Table).

• Fixed an issue where undo logs could not be reclaimed after the undo log reclaim feature was enabled.

• Fixed a database connection failure due to a read timeout when establishing physical replication on a read-only node.

• Fixed a node crash that occurred when executing an ADD INDEX or DROP INDEX statement after running ALTER TABLE ... DISCARD TABLESPACE in a Multi-master Cluster (Database/Table).

Category

Description

Release date

New features and performance optimizations

• The columnstore index feature now supports the MySQL DECIMAL data type.

• You can now sort data by a specified column when creating a columnstore index to improve pruning efficiency.

• The columnstore index feature now supports scanning data from files in ORC or PARQUET format stored in OSS.

• Cost-based optimization supports pushing down the outer join conditions of a materialized derived table into the derived table.

• Improved the optimizer's ability to estimate the number of scanned rows for join columns in equivalence derivation scenarios.

• The optimizer's join elimination feature now supports eliminating the inner table of a left join, including inner tables with multi-level nested joins and semi-joins.

• Supports the CAST(... AS INT) syntax, which behaves identically to CAST(... AS SIGNED INT).

• Multi-master clusters now support the multi-master mutual backup feature.

2023-02-21

Bug fixes

• Fixed a performance degradation issue in long-running transaction scenarios.

• Fixed a compatibility issue between Global Consistency (High-Performance Mode) and Parallel Query (PQ).

• Fixed an issue where using the transparent tablespace transport feature in a multi-master cluster would block DDL operations on other nodes.

• Fixed an issue where the free size of a table could, in rare cases, be displayed as 4 PB.

• Fixed a startup failure that occurred when multiple primary nodes of a multi-master cluster were restarted in parallel.

• Fixed a cluster crash that occurred when switching the write point for full-text indexes and columnstore indexes in a multi-master cluster.

• Fixed a database cache access anomaly when performing a copy DDL on a table with a foreign key.

• Adjusted physical replication connection timeout parameters to prevent stalls during physical machine failures.

• Fixed an issue where table locks could not be recovered during multi-master cluster crash recovery.

• Fixed a potential node crash when accessing the information_schema.tables table while adding a new node.

• Fixed a dirty page loss issue during the multi-master cluster crash recovery phase.

• Fixed a cluster crash that occurred when switching the write point after a database was dropped in a multi-master cluster.

• Fixed an invalid memory access issue in the TempTable engine when the thread pool was enabled.