ApsaraDB for ClickHouse:Release details of ApsaraDB for ClickHouse Enterprise Edition V24.8

New features

• The full_sorting_join algorithm is supported for the ASOF JOIN operation to improve the efficiency of sorting-based JOIN operations.

• The tuple function is extended to support queries that use named tuples.

• Functions are added, including tupleNames, fuzzQuery, percent_rank, changeYear, changeMonth, changeDay, changeHour, changeMinute, changeSecond, and groupConcat.

• Directory-related operations are simplified. When the file table function is used, the system automatically appends the asterisk wildcard character (*) to the directory path.

• The readWKTLineString function can be used to read MULTILINESTRING data in the WKT format.

• A configuration is introduced for Parquet files to enable or disable page index writing.

• Hive-style partitioning is introduced, which supports various storage engines, such as File, URL, S3, AzureBlobStorage, and HDFS, enabling efficient query and management for large datasets.

• The printf function is added to support Spark-compatible formatted output.

• A virtual column named _etag is added to the S3 table engine, which improves the processing efficiency of S3 data.

• A namespace mechanism is introduced for the query cache to consider identical queries with different tags as separate cache entries.

• The system table named system.error_log is added to record historical error information in the system.errors table and periodically write data to disks.

• The --memory-usage client option is added to monitor memory usage in non-interactive mode.

Performance optimization

• The Variant data type for binary serialization is optimized by enabling the compact mode to reduce redundant data storage.

• The data access performance of the S3 table engine is improved to speed up queries.

Breaking changes

• The CREATE MATERIALIZED VIEW ... ENGINE Replicated*MergeTree POPULATE AS SELECT syntax cannot be used in replicated databases.

• The alternative syntax for the LowCardinality data type that uses the WithDictionary suffix in previous versions is deprecated. We recommend that you update your table schemas to use the standard LowCardinality data type.

Fixed issues

• The following issue is fixed: When the Buffer storage engine is used with the destination distributed tables, query failures may occur, especially when the same table is referenced multiple times in a query.

• The following issue is fixed: When random distribution functions based on the Gamma function, such as Chi-Squared, Student, and Fisher, use parameters with near-zero negative values, computation duration is extended or infinite loops occur.

• The following issue is fixed: When the arrayWithConstant function processes large arrays, performance issues occur. The maximum size for each array needs to be limited to 1 GB.

• The following issue is fixed: Format issues occur in low-level memory management. The stability of memory allocations needs to be enhanced.

• The following issue is fixed: When the destination distributed tables and the Buffer storage engine are used, compatibility issues occur.

• The following issue is fixed: When the IS NULL condition is used in join operations, optimization errors occur.