PolarDB:PolarDB for MySQL 8.0.2 release notes

Description

• Added support for adding read-only column store nodes to a secondary cluster in a Global Database Network scenario.

• Added support for REORGANIZE PARTITION and LIST DEFAULT HASH partition to merge a LIST partition into a HASH partition, and split a VALUE into a new LIST partition.

  Added support for using the prefix of a HASH partition as the entire HASH partition name.

• Added support for large wide tables by extending the maximum number of columns for tables in the X-Engine to 10,000.

• Added the DISTANCE expression, which supports calculating cosine similarity, dot product similarity, and Euclidean distance.

• Added a feature to use IMCI to accelerate vector retrieval queries.

• Added support for the table value constructor syntax. You can now use the VALUES ROW syntax in DML statements.

• Added support for using a stored procedure to batch add columnstore indexes to all tables in a database.

• Added a feature in Outline to support table partitioning scenarios. A single Outline can match all statements with the same template, applicable to different partitioned tables.

• Added collection and use of non-index column statistics, and a unified management method for extended statistics.

• Added support for dynamically adjusting Order Index selection results at runtime based on actual conditions.

Description

• Optimized the startup speed of read-only column store nodes. The column store engine can now start independently from the column store snapshot offset.

• Optimized the startup time of column store nodes in scenarios with a massive number of tables.

• Optimized the Orca feature to support specifying a username and password in the Redis AUTH and HELLO commands using the username:password syntax.

• Optimized DDL scenarios for the X-Engine storage engine. The system will automatically enable the pre-commit feature for X-Engine.

• Adjusted the lock granularity for the sorted set data type in Orca. Only flags are used for lock checks on secondary indexes.

• Optimized the code for thread startup and destruction in logical pre-reading and simplified the memory check logic.

Description

• Fixed an issue where DDL operations on a table with a view could report an error or not take effect after a high-availability (HA) switchover.

• Fixed an issue where the parallel query parameter pq_enable_xengine_mpp did not take effect when set to OFF.

• Fixed an issue where the Autoinc value of a partitioned table occasionally rolled back, causing a Duplicate key error on INSERT.

• Fixed the session-level binlog_rows_query_log_events permission to support the execution of SQL backups and rollbacks in DMS.

• Fixed an issue where an error was not reported for duplicate items when setting a flagset-type variable, such as optimizer_switch.

• Fixed an issue where enabling the win magic switch caused a subquery rewrite overflow due to insufficient base_ref_items space, resulting in a process crash.

• Fixed an expression evaluation error when converting an INLIST to a JOIN result set if the INLIST VALUES contained a null value.

• Fixed an issue where using a HASH JOIN in the plan could cause incorrect parallel query results.

• Fixed an issue where incorrect results could be returned when Lateral Derived Tables (LDT) were present in a query.

• Fixed the implementation of the Redis protocol transaction model in Orca, including the WATCH, UNWATCH, MULTI, EXEC, and DISCARD commands.

• Fixed an issue where the results returned by the BITOP, ZRANGEBYLEX, ZRANGEBYSCORE, ZREVRANGEBYSCORE, and ZREVRANGEBYLEX commands in the Orca feature were inconsistent with official Redis.

• Fixed an issue that could cause a process to crash when the SUBSTRING_INDEX expression processes an empty string from GROUP_CONCAT and the separator's length is greater than 1.

• Fixed an issue where the CAST(REGEXP_SUBSTR(...)) AS DECIMAL statement caused a process to crash.

• Fixed an issue where data validation was not performed when changing the dimension of a VECTOR type with a DDL statement.

• Fixed a data dictionary inconsistency that occurred during a minor version upgrade from a version that did not support vector retrieval to one that did.

• Fixed a rare issue of duplicate data on the primary key index when using partition MDL.

• Fixed compatibility, scale-out, and scale-in issues between IMCI Serverless and internal memory elasticity.

• Fixed an issue where using the ALTER TABLE DROP statement to simultaneously delete multiple full-text indexes (FTS) could cause residual metadata.

• Fixed an issue where the uniqueness of index values was not leveraged for acceleration when a unique index was used.

• Fixed a cluster crash that occurred in extreme cases when performing DML operations on a table with an auto-increment column after online partition maintenance (MDL) was enabled.

• Fixed a potential cluster crash when a multi-value index was selected in a Multi-Range Read (MRR) optimization.

• Fixed an issue where Orca did not lock the database when executing certain management commands, such as the ORCA_OPTIMIZE, AUTH, and ACL commands.

• Fixed an issue where using a parallel query could lead to incorrect results when querying data that includes BLOBs.

• Fixed an issue where certain types of queries were not correctly routed in adaptive execution.

• Fixed an issue in cold data scenarios where the metadata cache on a read-only node was not updated after archiving to CSV or ORC format.

• Fixed a cluster crash due to a memory issue during DDL operations on a table containing a BLOB field.

• Fixed an issue where rebuilding an IMCI was not supported online, which made the columnstore index unavailable.

Description

Fixed an issue where an INSERT ... ON DUPLICATE KEY UPDATE statement caused a table's auto-increment value to roll back after an RO node switched to an RW node.

Description

• Optimized the statistics sampling mechanism by increasing the sampling frequency for large tables and ensuring coverage of the value range for each column. This improves the accuracy of the optimizer's cost estimation.

• Optimized the Serverless process for column store nodes.

  Allows a new column store node to serve requests only after its indexes are ready. This reduces slow queries caused by SQL statements being routed to the new node and running row store plans during this period.

• Optimized the performance overhead of saving partition information during SQL execution in IMCI, improving partitioned table query performance.

Description

• Fixed an issue where the result set was incorrect when merging IN keywords in a nested subquery.

• Fixed a crash that occurred when a related subquery used a nested window function.

• Fixed a crash during the migrate process caused by a concurrency issue between high-frequency INSTANT DDL operations and background compaction tasks.

• Fixed B-tree corruption caused by adding an auto-increment column and creating a reverse-order primary key index on that column.

• Fixed an issue where the Autoinc value could roll back to 0 after all data in a table was cleared.

• Fixed an issue where IMCI fell back to the row store when processing an expression that contained multiple subqueries with SUM.

• Fixed an issue where results could be incorrect when using a JSON_ARRAY expression in a subquery with IMCI enabled.

• Fixed a system crash that could occur during query transformation if the number of tables exceeded 61.

• Fixed an error when re-importing the same data into a UNIQUE CHECK IGNORE partitioned table using LOAD DATA with REPLACE.

• Fixed an Autoinc rollback issue in a partitioned table after an HA switchover.

• Fixed an issue where a process was interrupted after multiple rounds of restoration on the same table in the new database and table restoration process.

• Fixed an issue where rebuilding an IMCI was not supported online, which made the columnstore index unavailable.

• Fixed a rare issue where a query error was caused by accessing invalid data when the column store executed and accessed string type data.

• Fixed an issue where using Index_merge optimization for UPDATE statements could cause performance degradation.

• Fixed an issue that could be triggered when an Index JOIN in IMCI used a constant as a JOIN condition.

• Fixed an issue where table function condition pushdown caused execution failures during parallel execution.

• Fixed an issue where specifying a non-existent index in an Outline caused an error after multi-node ePQ was enabled.

• Fixed an issue where asynchronous DDL could not be used when adding an IMCI by modifying the COMMENT on a table with a full-text index.

• Fixed a potential abnormal cluster crash during the Undo initialization phase when Force Recovery was enabled.

• Fixed an issue where incorrect results could be returned in an InnoDB temporary table update scenario.

• Fixed an issue where a conflict between advanced transaction splitting in the Proxy and the Fast Query Cache affected the result set.

• Fixed an issue where parallel query results could be incorrect if the query contained a constant table.

• Fixed an issue where JSON_Length returned incorrect results when using a wildcard character.

• Fixed an issue where an RO node could get stuck during the HA phase, preventing prolonged HA times for users.

Description

Upgraded the file system version and optimized the process of restoring a cluster from a writable snapshot.

Description

Fixed a compatibility issue between the file system version and the control system.

Description

• Added support for the JSON_VALUE() function, which simplifies the operation of creating indexes on JSON columns.

• Added support for continuous business updates during Copy DDL execution.

• Added support for pushing down JOIN conditions to the corresponding views.

• IMCI adds support for an index join based on Hybrid Plan, which lets you use InnoDB indexes for JOIN operations.

• Added support for directly accessing AI node data tables through federated external tables.

• The OSS OUTFILE feature now supports parallel export to the OSS engine.

• Added a high-priority DDL capability to ensure successful DDL execution.

• Added a LOCK method to lock archived tables.

• Added a hybrid row-column optimizer and cost-based row-column routing capabilities.

• Added an optimizer feature for inner join or semi-join elimination.

  Optimized the outer join elimination feature and expanded it to more scenarios.

• Optimized large transactions in binary logging to avoid blocking other transaction commits.

• The Hybrid Plan in IMCI now supports partitioned tables, virtual columns, and columns of ENUM, SET, and GIS types.

• Added support for dynamically switching to the column store during the in-row execution process.

• The OSS query acceleration feature now supports archiving partitioned tables to OSS external tables. Manual or automatic partition archiving can synchronously generate OSS_FILE_FILTER query acceleration data.

Description

• Optimized PolarDB for MySQL 8.0.1 and later versions to support displaying outline call information in explain.

• Optimized the ORDER BY LIMIT pushdown feature for PolarDB for MySQL, supporting the pushdown of ORDER BY LIMIT to JOIN Tables.

• Optimized PolarDB IO Threads to be dynamically adjustable without requiring a restart.

• Optimized the time taken for query optimization of single-table queries.

• Optimized the DML transaction replay performance in IMCI.

• Optimized the IMCI bloom filter hash algorithm and reduced the overhead of building the bloom filter using string prefixes.

• Optimized the performance of simulated AIO.

• Optimized the low performance issue of 3AZ clusters under low concurrency.

• Optimized the nonblocking DDL feature to support more DDL statements.

• Optimized the memory usage of metadata information used in queries in IMCI.

• Optimized and adjusted the priority of PolarFS transaction processing threads to improve the DDL performance of PolarDB for MySQL.

• Optimized the performance stability of the file system when frequently creating and deleting table files during the DDL process.

Description

• Fixed a crash caused by a Hash JOIN between the same Common Table Expression (CTE) in a parallel query.

• Fixed an out-of-memory error reported during IMCI execution when a query had too many UNION ALL clauses.

• Fixed an issue where user threads on an RO node had to wait for a long time for a free page when the Buffer Pool was full.

• Fixed the issue of frequent sampling of large tables in the statistics sampling mechanism.

• Fixed an issue where removing an RO node when innodb_polar_log_rdma_transfer is enabled could block monitoring connections for a period of time.

• Fixed an incompatibility issue between some IMCI implicit conversions and the CASE statement in MySQL.

• Fixed an issue where global secondary indexes (GSIs) were not supported when archiving a partitioned table to an OSS external table.

• Fixed an issue where connection errors in the acknowledgment (ACK) packets of a secondary node could not be identified.

• Fixed an out-of-memory (OOM) error that occurred during the analyze table operation on a large table.

• Fixed an issue where useless subqueries were not deleted during the view merging process.

• Fixed an issue where during the inlist-to-JOIN conversion, an inconsistent inlist type caused inconsistent result sets.

• Fixed an issue where a query rewrite failed to read data when using index_merge to access a single-row subquery.

• Fixed an issue where converting a subquery resulted in a semi join crash.

• Fixed an issue where a query rewrite crashed when converting a scalar subquery with an anti join to a derived table.

• Fixed an issue of insufficient precision for floating-point numbers in JSON.

• Fixed an incompatibility issue between the global consistency high-performance mode (SCC) Log RDMA feature and GDN secondary clusters.

• Fixed an issue where converting a large double value to uint caused an overflow during the parallel computing planning process.

• Fixed an issue where the REF or MULTI RANGE access cost was underestimated for secondary indexes that did not include the partition key in a UCI partitioned table.

• Fixed an issue where the precision of a double value in JSON was abnormally lost during the parsing phase.

• Fixed a bug where the DB would get stuck during a manually triggered checkpoint.

• Fixed a crash caused by the interaction between the derived merge query rewrite and the scalar subquery unnesting rule.

• Fixed a security issue in the crash recovery process of MySQL external XA transactions.

• Fixed an issue where statistics on an IMCI partitioned table became invalid in a partition pruning scenario.

• Fixed an issue where cold data archiving failed for InnoDB tables with a function index.

• Fixed an issue where an incorrect delimiter setting resulted in an error instead of a warning when you set the values for optimizer_switch and parallel_query_switch.

• Disabled the index merge intersection access path by default for DML statements to improve performance and prevent deadlocks.

• Fixed an issue where the optimizer could not select the optimal JOIN ORDER due to cardinality estimation errors.

• Fixed an error when the CREATE TEMPORARY TABLE SELECT STATEMENT attribute had a column with column store.

• Fixed a potential assertion failure during the table optimization process.

• Fixed an issue where the query result cache could be inaccurate when time and datetime types are compared during single-node and multi-node parallel execution.

• Fixed an incorrect IP address in the audit log for connection reuse under the session-level connection pool.

• Fixed an issue where the cost of point queries and range queries based on the primary key was underestimated for UCI partitioned tables.

Description

• Optimized the memory usage of metadata information used in queries in IMCI.

• Optimized the memory footprint of the IMCI optimizer for high-concurrency query scenarios.

Description

• Fixed an error when creating a GSI on an empty table using parallel DDL.

• Fixed an issue of incorrect statistics estimation for large tables with Blob fields.

• Fixed an issue where the number of rows scanned by a derived table was not recorded in the audit and slow query logs.

• Fixed a space leak issue caused by an exception during the Shrink process.

• Fixed an error that occurred during a parallel query on a hybrid partition that contained both InnoDB and ORC partitions.

• Fixed an issue where the result set was incorrect when a subquery was incorrectly collapsed.

• Fixed an issue where an ADD COLUMN operation could not be performed on a partitioned table with subpartitions after upgrading the kernel version from 8.0.1 to 8.0.2.

• Fixed a potential cluster crash in a parallel query when a predicate condition contained a DEFAULT expression.

• Fixed an issue of excessive permissions during the execution of a DLM policy.

• Fixed a failure when archiving an archived table that had a descending index.

• Fixed an issue where the Blob space of the original record could not be reclaimed when updating a Blob column.

• Fixed an issue where IMCI could not be used because of inaccurate cardinality estimation when a BIGINT column with large values was used as a JOIN condition.

• Fixed an issue where statistics for an IMCI partitioned table could be inaccurate because only some partitions were sampled when updating the statistics.

• Fixed an issue where useless subqueries were not deleted during the view merging process.

• Fixed a potential parallel query crash after a CTE, which was the right table of a LEFT JOIN, was eliminated.

• Fixed an issue where the transaction ID for read-write transactions in SQL Explorer could be 0 when the thread pool was enabled.

Description

Added returned row count information to the Fast Query Cache feature. The number of returned rows for a cache-hit query can now be displayed in the audit log.

Description

• Fixed an issue where the number of rows scanned by a derived table was not recorded in the audit and slow query logs.

• Fixed an issue where no data was returned when Limit Offset pushdown was combined with an equality condition on a secondary index and an IN list condition on the primary key.

• Fixed an issue where the result of a comparison between a FLOAT type and an INT type could be incorrect.

Description

• Added the new parameter limit_orderby_pushdown_mode to control whether to push down `LIMIT` and `Order By` to derived tables or all sub-branches of a `UNION ALL`. When the `LIMIT` value is less than the value of the limit_pushdown_to_union_threshold parameter, `LIMIT` and `Order By` are pushed down to all sub-branches of the `UNION ALL`.

• Added a Left Join feature based on TopK pushdown in IMCI.

• Added an optimizer query transformation feature that can convert a LEFT JOIN to an INNER JOIN based on HAVING conditions and outer conditions of a VIEW.

• Added an optimizer feature to eliminate redundant DISTINCT operators in query statements to improve query execution speed.

Description

• Supports using the query acceleration feature for CSV format data files on OSS.

• You can execute the INSTANT DROP PARTITION operation on a partitioned table with a global secondary index (GSI) without invalidating the GSI.

• Optimized and improved the read performance of read-only (RO) nodes.

• Supports using the LEFT JOIN elimination feature in UPDATE statements.

• Optimized the SQL Trace feature to correctly record SQL statements and their execution plans when calling built-in stored procedures.

• Supports using the instant add column feature on tables with a columnstore index created.

• Supports using non-equivalent predicate derivation techniques to push down more predicates.

• The usage of the parameters related to automatic request distribution among row store and column store nodes is simplified. In addition, when the column store is not used, you can use SET use_imci_engine=FORCED to obtain a specific reason.

• Optimized the subquery execution process to collapse multiple subqueries as much as possible to eliminate them.

• Supports using temporary tables when using the Hybrid Plan to accelerate wide table queries.

• Optimized the speed of the Thread Pool in handling connection requests in low-concurrency scenarios.

• Supports using the rman.import command for columnstore indexes.

• Optimized the X-Engine sampling statistics feature to estimate the approximate compression ratio.

• Supports creating Tair read-only (RO) nodes to provide Cache read capabilities.

• When you monitor database performance, ignore the impact of access by system O&M engineers on the slow_queries status metric.

• Reduced the overhead of metadata for memory allocation and memory statistics in IMCI to save memory.

• Supports using the parallel query feature for single tables and partitioned tables in X-Engine.

• IMCI supports using the BatchProbe optimization method to improve the performance of JOIN operations.

• Optimized the memory of metadata, file modules, and small RAM resident objects in IMCI to reduce resident memory usage.

• Supports using the columnstore index and global consistency (high-performance mode) features at the same time.

• Stats Manager supports automatically dispatching the task of collecting histograms to read-only nodes or hot standby nodes for execution, to achieve automatic updating of histograms.

• Optimized the IO performance of PolarFS read-only nodes to improve their query response speed.

Description

• Fixed an issue where the automatic cancellation of a previous scale-in operation in the Buffer Pool could not be executed because a large user SQL query occupied pages and did not release them, preventing subsequent scale-out requests from being executed.

• Fixed an issue where the dbms_oss.delete_table_file built-in stored procedure could not delete cold data in CSV or ORC formats.

• Fixed a potential node crash caused by the automatic cancellation of a previous scale-in operation in the Buffer Pool under high concurrency.

• Fixed an issue where a query could not be dispatched to the columnstore index for execution if it contained a subquery that returned an empty result set.

• Fixed an issue where an internal counter error could occur in the Thread Pool if an ACL lock wait was encountered during the asynchronous authentication process.

• Fixed an issue where an assert message was returned when performing a multiplication operation on Decimal type data, if at least one multiplier was 0 and the scale (decimal places) of the result exceeded 16.

• Fixed an issue in the one-click upgrade scenario from RDS for MySQL to PolarDB for MySQL where a PolarDB for MySQL cluster restart interrupted the migration link.

• Fixed a node crash when querying a hybrid partition that contained both InnoDB and ORC partitions.

• Fixed data loss when repeatedly archiving data to an OSS external table.

• Fixed a node crash caused by accessing an invalid Blob reference when pre-reading Blob type data.

• Fixed a probabilistic node crash when the pre-read feature was enabled while using the columnstore index feature.

• Fixed an issue where the HINT syntax was not supported in the join condition pushdown feature.

• Fixed an issue where the HINT syntax was not supported in the predicate pushdown subquery feature.

• Fixed an issue of incomplete HAVING condition pushdown.

• Fixed a potential node crash in a parallel query when referencing a projection column in a HAVING clause and executing a non-correlated subquery.

• Fixed an issue of excessive Autoinc init log printing.

• Fixed an issue where the subquery merging feature did not support Optimizer HINT.

• Fixed a potential error in an OSS external table in an Order By Desc scenario.

• Fixed an out-of-bounds memory issue when compressing an oversized strpack in IMCI.

• Fixed a potential performance regression when pushing down conditions to a derived table.

• Fixed an issue where BIT fields were displayed incorrectly after a Group By.

• Fixed a slow query performance degradation issue caused by the impact of strict concurrency control of background compaction tasks in X-Engine on foreground operations.

• Fixed an issue where results could be incorrect in a parallel query if a plan segment had multiple tables performing a partition-wise parallel scan, and a parallel hash join operation was also executed in the segment.

• Fixed an issue where pulling up some subqueries caused a column in group statement is ambiguous warning.

Description

• Optimized the error message content for importing cold data from OSS back to the InnoDB engine.

• Supports configuring the tracking of slow query execution information in SQL Trace.

Description

• Fixed a memory leak that occurred when performing a Reload operation for features such as Statement Outline and Concurrency Control.

• Fixed an issue where performing a LEFT JOIN on tables in the INFORMATION_SCHEMA database returned incorrect results.

• Fixed an issue where a DLM policy could still take effect after the table structure of a table with a created DLM policy was changed.

• Fixed an issue where executing the INSERT INTO ON DUPLICATE VALUE command on a partitioned table reported the Can't find record on <table> error.

• Fixed a rare crash that occurred when executing the TRUNCATE PARTITION command on a partitioned table with an auto-increment column.

• Fixed an issue where the number of parallel shards could be too small, or the data of concurrent workers could be unbalanced, when performing a parallel query on a large table with a high B-tree or skewed B-tree data.

Description

• Supports granting users global dynamic permissions to execute the SHOW_ROUTINE, FLUSH_USER_RESOURCES, FLUSH_TABLES, FLUSH_STATUS, and FLUSH_OPTIMIZER_COSTS commands.

• Disabled the small partial update feature for Blob to avoid potential data read errors when reading Blob type fields.

• When you use the HINT syntax SET_VAR to set the value of the loose_optimizer_switch parameter, you no longer need to add a comma at the end.

Description

• Fixed an issue where an index was corrupted and table data could not be accessed due to different index orders when executing an EXCHANGE PARTITION operation.

• Fixed a deadlock issue in DDL read-ahead in parallel DDL scenarios.

• Fixed an issue where the LSN offset of a checkpoint did not meet expectations during a checkpoint operation, causing the crash recovery logic to fail to run normally after an abnormal cluster restart.

• Fixed an issue where a node restart was required to recover from an Autoinc rollback.

• Fixed an issue where a table definition has changed error occurred when a Hybrid Plan was applied to an SQL statement that includes an `Order By` clause and returns `NULL` values.

• Fixed an issue where an error was not correctly reported when the number of input variables for some built-in stored procedures was incorrect.

• Fixed an issue where duplicate groups appeared in the result set when a parallel query pushed down GROUP BY to a Worker for parallel execution on a partitioned table.

• Fixed an issue where the “Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs” error was incorrectly reported when you create a table that contains a column of the DECIMAL type.

• Fixed a rare crash during the undo cleanup process when using a temporary table after the backquery machine flashback query feature was enabled.

• Fixed an issue where the optimizer selected an ordering index, but the sort operation was not omitted during actual execution.

• Fixed an issue where the column information of the remaining tables was not updated after using the JOIN elimination optimization feature, which could prevent the selection of the optimal access method for the table.

• Fixed a potential error in the query result when Semi join selected the Materialization policy.

• Fixed an issue where a Binlog record failed to uniquely identify a record in a partitioned table whose primary key does not contain the partition key when binlog_row_image=MINIMAL.

• Fixed an issue where a UCI index might not be unique when the primary key did not include the partition key in the following scenarios:

  • Pulling up an inner table of a semi-join;

  • GROUP BY or DISTINCT elimination;

  • Adding a unique key dependency for JOIN elimination;

  • Decorrelating a correlated scalar subquery by converting it to a derived table through a window function;

  • The columnstore index is optimized for GroupJoin query scenarios.

Description

• After enabling the global consistency (high-performance mode) feature, if the currently executed SQL statement does not need to use this feature, you can use the Hint syntax in the current SQL statement to disable it.

• Supports using correlated subqueries in the Grouping clause and eliminating complex correlated subqueries.

• Supports using Concurrency Control and Statement Outline in a Multi-master Cluster (Database/Table). When you add or delete Concurrency Control and Statement Outline on a primary node of a Multi-master Cluster (Database/Table), the other primary nodes in the cluster will automatically synchronize the addition or deletion.

• Supports adding a Tair cache node to a PolarDB for MySQL cluster to read data from the PolarDB for MySQL database.

Description

• Optimized the speed of the Thread Pool in handling connection requests in low-concurrency scenarios.

• Optimized the startup process of IMCI to reduce the startup preparation time for the column index.

• Improved the data read performance of read-only (RO) nodes in scenarios where a secondary index looks up data from tables based on the primary key.

• Optimized the transaction mask calculation process in IMCI.

Description

• Fixed an issue where CCL rules matched by DIGEST value could become invalid after a cluster upgrade.

• Fixed a node crash caused by adding backticks to the input parameters when calling some built-in stored procedures.

• Fixed an incorrect sorting result when querying TopK data using the IMCI feature, due to too many NULL values in the tables involved in the query statement, when sorting in descending order.

• Fixed an issue where a read-only column store node could not recover column store data when it was started during the execution of a DDL on a table involving a columnstore index.

• Fixed an issue where the elastic parallel query (ePQ) feature could not be used in query statements containing a Recursive CTE in some special scenarios.

• Fixed a failure to add a new RO node due to a low registration offset of the RO node when executing the new database and table restoration process.

• Fixed an issue where a thread could not exit due to a full disk when executing the new database and table restoration process.

• Fixed an anomaly on a read-only node caused by concurrent modification of the data structure during physical replication in the X-Engine.

• Fixed an upgrade failure when upgrading an X-Engine from a read replica to a standby.

• Fixed an issue where data was not re-partitioned when modifying the character set of a partition key in a partitioned table.

• Fixed an issue of incorrect query results when querying a view containing BIT type data using IMCI.

• Fixed an issue of high latency when performing write operations on a table in the X-Engine under low concurrency.

• Fixed an issue where GROUP BY could not find the corresponding column after a single-row subquery was pulled up.

• Fixed an issue of incorrect result set after MySQL community enabled the subquery_to_derived feature.

• Fixed an issue where using the polar_abbrev_sort_key_size parameter to control sorting operations caused memory corruption.

• Fixed an issue where only data from the first partition was returned in an index merge scenario that selected a unique key (UK) when performing an equality query on a partitioned table whose primary key did not include the partition key.

Description

• For clusters that have not had a Statement Outline added, if a Statement Outline is added after a version upgrade, outline matching will no longer be affected by the character set.

• Optimized the process of enabling global consistency (high-performance mode).

• Supports accessing the INFORMATION_SCHEMA view information of the column store from any node. By default, it aggregates and displays information from all column store nodes.

• Optimized the database and table restoration feature. For example, it improves the restoration speed and supports task failure rollback.

Description

• Fixed a potential node crash when performing a parallel scan on a derived table that contained a ROLLUP calculation.

• Fixed a node crash caused by an incorrect format of a user-defined CONNECTION when creating an OSS external table.

• Fixed an issue where the RW node did not interrupt its replication relationship in special cases when the physical machine of an RO node failed, which caused the dirty page flushing of the RW node to be blocked.

• Fixed an Autoinc rollback issue caused by table cache eviction due to an excessive number of tables after an HA switchover.

• Fixed a replica node crash caused by an extreme Redo log sequence generated from a concurrency relationship between deleting a table or index in X-Engine and an occasional background SwitchMemtable operation.

• Fixed an internal connection object leak caused by an abnormal execution of elastic parallel query (ePQ) when a remote connection was forcibly closed.

• Fixed an incorrect query result when executing a ROLLUP statement in parallel, due to multiple identical constant columns in the GROUP BY clause.

• Fixed an ASSERT CRASH (assertion error) triggered when accessing a GSI of a partitioned table using RANGE INDEX, in a case where all partitions of the partitioned table were pruned during the partition pruning phase.

• Fixed an archiving failure when archiving cold data to ORC format, caused by a Blob type field in the table.

• Fixed a restriction issue of parallel query on a GSI in MRR and other scenarios.

Description

Introduced a patch from the official MySQL 8.0.16 to fix a deadlock caused by an oversized Blob.

Description

• A new parameter, opt_non_covering_key_cost, is added to control whether the initial seek cost of a secondary index is considered.

• Added the parameter worst_seek_records_threshold to control whether to limit the maximum I/O cost of equality lookups on non-covering indexes during cost estimation.

Description

• Optimized the logic for switching a read-only (RO) node to a primary node.

• Optimized the logic of the Thread Pool for handling new connections.

• Supports pushing down conditions to a materialized table when the subquery is a UNION clause.

• Supports pushing down as many pushdown-able conditions as possible to a materialized table through the transitive relationship of equality conditions.

• The optimizer supports pushing down eligible predicates into a subquery.

• Optimized the cost model for equivalent non-covering index access.

• Optimized memory by reducing the memory consumption of table object cache and table definition cache.

• Supports using the DLM automatic cold data archiving feature for all RANGE type partitioned tables.

• Supports the optimizer selecting a GSI of a partitioned table in a query with partition pruning.

Description

• Fixed a potential cluster startup failure when the AccessKey information of the OSS SERVER was corrupted.

• Fixed a query failure in some scenarios due to the GroupJoin operator occupying too much memory.

• Fixed an issue where some queries with OR predicates could not be executed because the query plan could not bind to the correct columns.

• Fixed an issue where the optimizer could not accurately estimate the Join selectivity when joining an unsigned column with a signed column.

• Fixed a potential database restart when a query was terminated by the scheduler.

• Fixed a space reclamation failure caused by a read-only (RO) node of the X-Engine failing to report version reference information.

• Fixed a RowID rollback failure when loading a checkpoint that was ahead.

• Fixed an IO performance degradation issue in the columnstore index in a late materialization scenario.

• Fixed a potential infinite loop issue during partition pruning if a partition was not located.

Description

Fixed a potential node crash when deleting the last CCL rule of the same SQL statement type, where only the keyword was filled in but not the schema and table.

Description

Improved the speed of archiving cold data to OSS in CSV format.

Description

• Fixed an issue where a Table definition has changed error might be triggered when the global consistency (high-performance mode) and elastic parallel query (ePQ) features are enabled at the same time.

• Fixed an issue of inaccurate scan row count statistics for SQL statements in an Index condition pushdown scenario.

• Fixed a node crash when using the Phrase search mode in a full-text index.

• Fixed an issue where a newly connected RO node would crash when the read-write (RW) node finished an undo truncate.

Description

• Fixed an issue of high CPU usage by a background thread.

• Fixed a potential cluster startup failure when the AccessKey information of the OSS Server was corrupted.

Description

Added the innodb_polar_import_tablespace_strict_mode parameter. The default value is ON, which means an error will be reported if the table structure contains a full-text index when performing a database and table restoration operation.

Description

• Supports using the columnstore index to query data in specified partitions of a partitioned table.

• Supports using the primary index in the InnoDB engine to accelerate queries for the columnstore index.

• Supports converting an IN subquery to a Semi-Join subquery when performing an UPDATE or DELETE operation on a single table.

• In a Join query, supports converting a scalar subquery to a materialized table.

• Supports using the Fast Query Cache feature for partitioned tables in the InnoDB engine.

• An implicit auto-increment primary key is not automatically created when creating an OSS external table.

• Supports creating a GSI on a partitioned table with UNIQUE CHECK IGNORE=1.

Description

• Fixed a crash in the JPPD optimization feature when a CTE contained a nested CTE with multiple references.

• Fixed a restoration failure when performing a database and table restoration operation on a hybrid partitioned table.

• Fixed an issue where a secondary node did not trigger a reconnection mechanism when the primary node was shut down and restarted.

• Fixed a rare re-interruption of the process during fault recovery.

• Fixed an issue of inserting data into the wrong partition when inserting data into a UNIQUE CHECK IGNORE partitioned table.

Description

Added the ALTER TABLE table_name CHANGE PARTITION partition_name ENGINE=CSV/ORC FORCE; syntax.

Description

• Supports log transmission between read-write (RW) nodes and read-only (RO) nodes through an RDMA network.

• Supports archiving partitioned tables and standard tables with auto-increment columns to OSS.

• The federated query engine supports pushing down the Order syntax to a remote database. The Order syntax can be used with Limit Offset to reduce network overhead.

• Supports creating a columnstore index on a hybrid partitioned table (InnoDB+ORC).

• Optimized the performance of the Thread Pool in handling short-lived connections.

• The DLM feature supports archiving InnoDB partitions of a partitioned table to OSS.

• Supports deleting and adding InnoDB partitions in a hybrid partitioned table.

Description

• Fixed an issue where a query could not be promptly interrupted and temporary files could not be cleaned up after the temporary files used for a columnstore index query filled up the disk.

• Fixed an issue where parameters were not effective when a stored procedure was executed repeatedly.

• Fixed an issue where a regular user could delete sys.

• Fixed an issue where an abnormal internal counter in the Thread Pool could cause a backlog of SQL requests during concurrent recording of slow query logs.

• Fixed an abnormal node restart when performing a database and table restoration on a partitioned table after an HA switchover.

• Fixed an issue where the checkpoint LSN did not advance as expected.

• Fixed a potential error when executing a query on an RO node while an ALTER TABLE ... IMPORT TABLESPACE operation was being executed on the RW node.

Description

• Added the ALTER TABLE table_name CHANGE PARITION part_name ENGINE = 'csv'; command to archive partitions.

• Added the row-level compression feature.

• Added the primary key data slicing function call dbms_dts.show_parallel_ranges (<schema_name>, <table_name>, [<suggestion_ranges>]).

Description

• Supports creating a columnstore index on a virtual column.

• Supports using the recycle bin feature for tables with a created columnstore index.

• Optimized the OSS query capability. After the elastic parallel query feature is enabled, a parallel query is directly performed if the value of the system's loose_csv_max_oss_threads parameter is greater than 1.

• Optimized the error message content for incorrect OSS external table file formats.

• Supports reading data in ORC format from OSS and writing data in ORC format to OSS.

• Optimized the error message content for spelling errors.

Description

• Fixed a replication delay issue when replaying Redo logs of a wide table on an RO node.

• Fixed a node crash when using order index optimization to query a hybrid partitioned table.

• Fixed a node crash when truncating a hybrid partition.

• Fixed an incorrect query result issue when using the Limit Offset pushdown optimization feature with a query statement that contained a window function.

• Fixed an issue where OSS could not perform parallel execution.

• Fixed an issue where OSS could not read JSON files.

• Fixed a spelling error in the engine.

• Fixed an issue where only the first file might be read when reading CSV format cold data from OSS.

• Fixed an issue where data containing \n could not be read after being archived by DLM.

• Fixed an issue where temporary files might remain on OSS after cold data archiving.

Description

• Added the Hint types BNL, BKA, and MRR to Hint Plan.

• Added the variable by_expr_ignore_charset to SQL Sharing. After setting by_expr_ignore_charset to ON, the digest differences generated by statements with Order By or Group By under different character sets will be eliminated. However, after setting it to ON, all digest values generated in this mode will be different from the digest values generated before it was set to ON.

• The Statement Outline feature now has the interface dbms_outln.add_optimizer_outline('schema', 'hints', 'query') for adding Optimizer Hints.

• Added the ALTER TABLE table_name CHANGE PARTITION partition_name ENGINE = 'csv'; command to archive all data of a specified partition on a partitioned table to OSS.

Description

• Optimized memory consumption when scanning tables in the recycle bin.

• Supports querying gtid information on a read-only node.

• The columnstore index supports instant add column by rebuilding the columnstore index data.

• The columnstore index supports the REGEXPR function.

• The columnstore index supports pruning optimization based on judgment conditions related to IS NULL and IS NOT NULL.

• Supports using parallel DDL to create a GSI in parallel.

• Supports using the instant add column feature on a partitioned table with a created GSI.

• Supports converting a table with a created GSI to an interval range partitioned table, and also supports creating a GSI on an interval range partitioned table.

• When no Statement Outline is added, the Digest value of an SQL statement is no longer calculated when the SQL statement is executed.

• Nonblocking DDL supports using the OPTIMIZE TABLE and TRUNCATE TABLE statements.

Description

• Fixed a database connection failure due to a read timeout when establishing physical replication on a read-only node.

• Fixed an incorrect pushdown of a table with a NULL field when the engine_condition_pushdown parameter was set to true.

• Fixed an issue where the source file on OSS could not be deleted when executing the ALTER TABLE engine = csv storage oss, force; command.

• Fixed a database node crash when performing a check operation on an empty OSS external table.

Description

• Added the nonblocking DDL statement feature.

• Added the SQL Trace feature.

• Added the SQL detail feature.

• Added the Auto Plan Cache feature.

• Added the columnstore index feature.

• Global consistency (high-performance mode) now has the Innodb_scc_timeout_count parameter to record the number of timeouts during a query.

• Global consistency (high-performance mode) now has the Innodb_scc_timeout_degrade_count parameter to record the number of times a query timed out and was converted to an asynchronous query.

Description

• When using Statement Outline, if a specified Index Hint does not exist, a warning is reported instead of an error.

• The execution plan is now displayed in the extended data of the slow query log.

• Supports adaptive digest length, which means the memory space for statement digest calculation can be adaptively adjusted according to the length of the query statement.

• Concurrency Control (CCL) now has the ccl_mode parameter. When the database access traffic reaches the concurrency control limit, other requests to access the current cluster are rejected.

• Built-in stored procedures now support using variables starting with the at sign (@) as input values.

• Optimized the internal lock granularity of PolarFS, significantly improving performance in some high-load scenarios.

Description

• Fixed an issue where no error was reported when adding or deleting CCL rules or Statement Outlines on a hot standby node or a read-only node.

• Fixed a cluster crash caused by executing the SHOW SLAVE HOSTS statement on the primary node.

Description

Added the loose_group_by_compatible_sorting parameter to control whether to enable the Group By compatible sorting mode.

Description

A Statement Outline with an empty Schema_name now matches all SQL statements with the same Digest value.

Description

Fixed a potential illegal memory access when using a DDL statement on a partitioned table.

Description

• When a partition expression contains an AUTO INCREMENT column, you can now specify a value for that column, and partition pruning will be performed based on the specified value.

• You can use the without validation option in the add partition syntax to add a LIST partition to a LIST DEFAULT HASH partitioned table.

Description

The syntax for converting a standard table to a partitioned table now includes the without validation option. The syntax is as follows: Alter table t1 partition by range(a) (partition p0 values less than (100), partition p1 values less than (200)) without validation.

Description

• Cost-based calculation now supports pushing down the outer join conditions of a materialized derived table into the derived table.

• Supports converting a predicate condition with an IN-List to a temporary table, performing a semi-join with the query in the from clause, and avoiding the overhead of repeated materialization through a zero-copy method.

• Supports pushing down conditions from the having clause to the where clause.

• Subpartitions now support executing DDL operations under a partition-level MDL, reducing the lock granularity. During a DDL execution, only the modified partition is affected, and DML operations on other partitions are not affected.

Description

• When TDE is enabled, encryption of system tables is disabled. System tables are not allowed to be encrypted.

• Adjusted timeout parameters related to physical replication connections to avoid stuttering issues during a physical machine failure.

Description

• The semicolon at the end of the SQL statement in the query parameter of the dbms_ccl.add_ccl_digest_rule, dbms_outln.add_index_outline, and dbms_outln.add_optimizer_outline built-in stored procedures is now ignored.

• Changed the MEMBER keyword to a non-reserved keyword.

• Supports the Partial Result Cache feature.

• The optimizer's join elimination feature now supports eliminating the inner table of a left join (including inner tables with multi-level nested joins and Semi-Joins).

• The federated query engine supports pushing down all compatible conditions to the remote server for execution, and only returns the columns required by the query.

• The federated query engine supports pushing down the Limit Offset syntax to the remote database for execution.

Description

Fixed an issue where IF NOT EXISTS could not be used in the syntax for creating a Server.

Description

• Added the SELECT INTO OSSOUTFILE statement to export a local table (a table on PolarFS) to OSS.

• Supports the failover with hot standby feature.

• Added the dynamic_partition_prune_enabled parameter to control the dynamic partition pruning feature. When this feature is enabled, in multi-table join query scenarios involving partitioned tables, if the join condition meets the partition pruning condition, unnecessary partition scans can be reduced, thereby improving query performance.

Description

• Supports creating indexes on some partitions of a partitioned table, which means it supports creating, deleting, and rebuilding indexes on a partitioned table at the partition granularity.

• Supports performing REBUILD PARTITION and REORGANIZE PARTITION operations under a partition-level MDL, so that during a DDL operation on a certain partition, DML operations on other partitions are not affected.

• The username and password in the mysql.servers table are no longer stored and displayed in plaintext.

  Note

  The username and password of an existing Server will not be encrypted after an upgrade. You need to manually delete and recreate it.

• Supports using the partition-wise join technique between a partitioned table and a standard table, significantly improving the join efficiency of a partitioned table and a standard table with a large amount of data.

• Statement Outline supports all UNION clauses.

• Supports using the Limit Offset pushdown feature when accessing a specific partition in a partitioned table.

Description

• Fixed an issue of inaccurate estimation of the number of unique values for a GROUP BY field in the parallel optimizer.

• Fixed an issue where an SQL statement would record a Binlog during a hot upgrade.

Description

• Multi-node elastic parallel query (ePQ) supports cross-node queries of external table data on OSS.

• Supports using a GSI on a partitioned table.

• Supports using multi-stage multi-node parallel query in queries with SJM (Semi-join Materialization).

Description

• Fixed an ERROR 1028 (HY000): Sort aborted: Query execution was interrupted issue caused by the early termination of limit1.

• Fixed a potential node crash when writing an implicit primary key through a scheduled task.

• Fixed an incorrect result set issue with count(distinct ...) under multi-node elastic parallel query (ePQ).

• Fixed a potential incorrect query result set issue in a parallel query when the query condition contained DATE and DATETIME types.

Description

Fixed a crash issue on an RO node caused by a file name hash collision.

Description

• Thread pool optimization: When a database transaction is waiting for a metadata lock or row locks, the thread pool can generate additional threads to process new requests.

• Enhanced CCL to support DDL throttling.

• Accelerated the DDL data rebuilding process by asynchronously writing to a temporary file.

• Improved thread pool performance under high concurrency.

• Added the oss_sts_token parameter to the syntax for creating an OSS server, used to configure a temporary access credential for OSS.

Description

Fixed an incorrect result issue when a query had both Limit offset pushdown and MRR optimization.

Description

• Supports creating a hybrid partitioned table. For more information, see Create a hybrid partition.

• Read-only nodes support custom memory temporary tables (currently only the MEMORY engine and MyISAM engine are supported).

• Supports the ReadableProto feature. By specifying the Protobuf protocol, you can directly read information serialized and stored in a BLOB type field on the database.

• Optimized DDL sorting: Changed from two-way merge to multi-way merge to reduce the number of IOs and improve the performance of adding an index with DDL.

• Added the oss_prefix parameter to the syntax for creating an OSS server, used to configure the OSS file path.

Description

• Fixed an unordered result issue in concurrent execution when group_by_implicit_sorting selected an index for a group by column.

• Fixed an issue where cost and rows were incorrect in the Hash Join result displayed by the EXPLAIN FORMAT=TREE statement.

• Fixed an issue of excessive physical replication latency under low cluster load.

• Fixed an issue where the LRU List cleanup was blocked and could not release a Page when there were too many dirty pages in the Buffer Pool and flushing was slow.

• Fixed a database crash caused by an overly long record generated when rolling back an Update operation after an instant add column.

Description

Supports users creating and using Federated engine tables.

Description

• Fixed a potential incorrect result set issue with SKIP_SCAN when multiple sets of data existed in the index prefix.

• Fixed an incorrect result set issue with SKIP_SCAN after data in the database was marked for deletion.

• Fixed an issue where the fast query cache feature blocked the synchronization of redo logs when the read-only node obtained a metadata lock (MDL).

Description

Fixed an issue where Using index for group by returned an incorrect result.

Note

• PolarDB's parallel query supports a multi-node parallel execution engine. For a single query, some or all Workers can be scheduled to run on remote nodes. This process uses the computing resources of multiple nodes in the cluster. Details are as follows:

  • The degree of parallelism for a parallel query can exceed the bottleneck of a single node. This accelerates analytical queries, such as large queries.

  • Scheduling queries to idle nodes based on the global resource load accelerates the queries and improves resource utilization.

  • You can create multiple cluster groups to form a computing resource pool. This provides more flexible elasticity for computing resources.

• The parallel query parameter parallel_query_switch has a new switch restrict_on_limit_without_order, which controls parallel queries with a limit but no order by.

• PolarDB now supports condition pushdown to a materialized derived table.

Description

The SET INTERVAL syntax is added to modify the interval type and value of interval range partitioning. You can also use this syntax to transform a range-partitioned table into an interval range partitioned table, and vice versa. For more information, see Modify interval range partitioning and Transform a partitioned table.

Note

• OSS external tables are supported. For more information, see Access OSS data through an OSS external table.

Description

• Fixed an issue where the history list could not be deleted, causing the undo log to expand.

• Fixed a failure when creating a partitioned table with CREATE TABLE .. LIKE.. LIST DEFAULT HASH.

• Fixed a display issue of UNIQUE CHECK IGNORE in the system table mysql.tables.

Description

• Enhanced the Elastic Parallel Query capability. The linear acceleration capability has been upgraded, and a multi-stage parallel computing capability has been introduced.

• Enhanced partitioned table capabilities. More partitioned table types are supported, and Interval partitioning is supported. For more information, see Partitioned table.

• Supports subquery decorrelation.

• In conjunction with PolarProxy, supports defining sensitive fields through stored procedures and tagging the metadata of the values of these sensitive fields.

• Added the STATUS variable Online_altered_partition to count the number of times ALTER TABLE ADD PARTITION and ALTER TABLE DROP PARTITION operations are performed online.

• The database kernel supports transaction breakpoint resume and read-only nodes reading uncommitted transactions.

Description

• Supports the SELECT FOR UPDATE WAIT N and SELECT FOR SHARE WAIT N syntax. Here, N represents the timeout in seconds for a single row lock. If a query needs to lock multiple rows, the time is not accumulated. Timeout detection is performed only for a single row. If the wait time exceeds N seconds, a lock wait timeout error Lock wait timeout exceeded; try restarting transaction is returned.

• Fixed an issue where the optimizer did not choose a better index range path for some GROUP BY statements.

• Fixed an incorrect result from the GROUP_CONCAT function caused by an arithmetic overflow when the group_concat_max_len parameter had a large value.

• Supports modifying the database cache size.

• Avoids unnecessary traversal of partitioned table partitions to improve query efficiency.

• Fixed a DDL operation failure on an encrypted table.

• Optimized master_key_id_mutex to allow DDL operations to be executed in parallel.

• Fixed a replay log crash on a read-only node under high concurrent pressure.

• Fixed a crash at m_recv_bits.is_set(fold) on a read-only node during physical replication under high concurrent pressure.

Description

The default value of the innodb_adaptive_hash_index parameter has been changed from ON to OFF.

Note

The database cache size can be modified.

Description

Supports the official MySQL 8.0.20 Index Hints feature.

Description

• Fixed a stability issue with parallel Hash Join.

• Fixed an incorrect estimation of the number of rows corresponding to each index key in the statistics.

• Fixed an error when processing empty data while scanning a shared temporary table in parallel.

• Fixed a data processing error in the official REGEXP_REPLACE function.

• Fixed a crash in the official version for complex queries containing subquery constants.

• Fixed an issue where filesort returned incorrect data.

Description

• Enhanced the cost estimation model performance for Hash Join.

• Supports parallel Hash Join, including parallel Build and Probe phases.

Description

• Fixed an incorrect display of the number of parallel partitions in a parallel execution plan.

• Fixed an abnormal crash issue with a parallel subquery.

• Fixed an issue where multiple worker threads could not produce random results when using the RAND() function in a parallel query.

Description

PolarDB parallel processing now fully supports the latest Volcano model iterative executor.

Description

Optimized the write performance of the cluster.

Description

Fixed a memory leak issue.

Description

Adds the Warm Buffer Pool. Data in the Buffer Pool persists during crash restarts and upgrades. This feature significantly speeds up restarts and prevents performance degradation after a restart.

Description

• EXPLAIN now supports displaying parallel plans. Use FORMAT=TREE to view the parallel execution plan tree.

• Parallel query now supports hash join, which improves JOIN performance for queries that do not use an index. For more information, see Parallel acceleration of Hash Join.

• Adds support for the Resource Manager feature, which lets you monitor CPU and memory resources. For more information, see Resource Manager.

• Adds support for Performance Agent. This agent collects and aggregates internal performance data from nodes in a PolarDB for MySQL cluster. Query the PERF_STATISTICS memory table to obtain performance data for specific metrics. For more information, see Performance Agent.