ApsaraDB for ClickHouse:Release details of ApsaraDB for ClickHouse Enterprise Edition V24.10

New features

• The toStartOfInterval function is added to support time alignment with a custom starting point and enhance compatibility with the time bucketing feature of TimescaleDB and PostgreSQL. Date or timestamp values can be aligned to a specific interval. 

• The arrayZipUnaligned and arrayElementOrNull functions are added to improve Spark compatibility and the flexibility of array operations.

• The tupleElement(expr(), 'name') syntax is supported. Named tuples can be processed by using expressions in the expr().name format, enhancing JSON processing capabilities.

• Refreshable materialized views are supported in replicated databases, facilitating the use of materialized views.

• The Dynamic data type is refactored. When the limit of the Dynamic data type is reached, new data types are not forcefully converted to the String type but stored in the binary format as separate subcolumns. 

• The MongoDB driver is upgraded to the new mongocxx drive. The upgrade removes support for the old protocol, adds support for all MongoDB data types, and enables the execution of the WHERE and ORDER BY clauses on the MongoDB side.

• Query caches can be deleted by tag. Example: SYSTEM DROP QUERY CACHE TAG 'abc'.

• Support for reading Iceberg tables is enhanced, and compatibility with Iceberg data sources is improved.

Performance optimization

• Optimized query progress display: The progress-table parameter is added to enable real-time display of changing metrics during query execution. You can press the Space key to display or hide a real-time progress table.

• Improved JSON format support: The input_format_json_empty_as_default setting is added. When you enable the setting, the setting considers empty fields in JSON inputs as default values, improving the flexibility of JSON processing.

Breaking changes

• Adjustment of SETTINGS in UNION queries: The SETTINGS setting can be added before FORMAT in a UNION query with subqueries. If multiple SETTINGS settings are added, the setting that is closest to a subquery takes effect.

• Adjustment of the Dynamic data type: When the limit of the Dynamic data type is reached, new data types are not forcefully converted to the String type but stored in the binary format as separate subcolumns.

Fixed issues

• The following issue is fixed: When optimize_functions_to_subcolumns is enabled, a column type error occurs, and the LowCardinality data type cannot be processed.

• The following issue is fixed: SQL queries prematurely terminate due to inaccurate memory statistics.

• The following issue is fixed: When empty fields are used, data in the JSON format cannot be parsed.

• The following issue is fixed: Query caches cannot be accurately deleted in specific cases.

• The following issue is fixed: The lack of support for sparse delete operations in MergeJoin can result in data merging issues.