ApsaraDB for ClickHouse:Release notes

Feature

Description

References

V22.8

• Optimization

  • When you import data from an external table in MaxCompute, support for data compression during network transmission and exclusive Tunnel is added.

  • By default, text_log is enabled. Logs whose log levels are Warning, Error, and Fatal are recorded. You can query the logs by using system.text_log.

  • query_thread_log is disabled by default to resolve the issue that a large amount of disk space is occupied.

• Fixed issues

  • The issue of data type conversions and value comparisons between DateTime64 and Datetime or between DateTime64 and Date is fixed.

  • The following issue is fixed: "Invalid number of rows in Chunk".

Import from MaxCompute

V21.8

• New feature

  A fallback session lifetime mechanism is introduced to connect ApsaraDB for ClickHouse to ZooKeeper. This mechanism balances loads between ZooKeeper nodes.

• Fixed issue

  The out of memory (OOM) issue caused by index_granularity=0 is fixed.

None

Feature

Description

References

Newly regional availability

New feature

ApsaraDB for ClickHouse clusters are available in Malaysia (Kuala Lumpur).

• Create an ApsaraDB for ClickHouse cluster

• Fees of compute nodes in ApsaraDB for ClickHouse clusters

• Storage fees of ApsaraDB for ClickHouse clusters

• Fees for backup space in ApsaraDB for ClickHouse clusters

V22.8

Fixed issue

The following issue is fixed: The query result of the Date function is abnormal.

None

Feature

Description

References

Data backup disabling

New feature

The data backup feature can be disabled for an ApsaraDB for ClickHouse cluster.

Data backup

Feature

Description

References

Scheduled restart of clusters

New feature

An ApsaraDB for ClickHouse cluster can be restarted at a scheduled time.

Restart an ApsaraDB for ClickHouse cluster

Tag management

New feature

ApsaraDB for ClickHouse allows you to add tags to an ApsaraDB for ClickHouse cluster.

Tag management

ZooKeeper configuration

New feature

By default, ApsaraDB for ClickHouse clusters provide ZooKeeper with 4 cores and 8 GB of memory for free.

None

Feature

Description

References

Monitoring and alerting

New feature

View monitoring data in the ApsaraDB for ClickHouse console

Upgrade of the major engine version

New feature

The major engine version of ApsaraDB for ClickHouse clusters can be upgraded by migrating data between the clusters.

Upgrade the major engine version

Feature

Description

References

Release and mounting of a Classic Load Balancer (CLB) instance

New feature

A CLB instance can be released and mounted for an ApsaraDB for ClickHouse cluster that is created after December 1, 2021.

Release and mount a CLB instance

Account management

New feature

A database account of an ApsaraDB for ClickHouse cluster can be configured by executing SQL statements.

• Manage database accounts of an ApsaraDB for ClickHouse cluster

• CREATE USER

• GRANT

Feature

Description

References

V22.8

• New features

  • The time range of Date32 and DateTime64 is expanded. The valid value ranges from 1900-01-01 to 2299-12-31.

  • Lightweight delete is supported.

  • SQL user-defined functions and executable user-defined functions are supported.

  • Partition-level preprocessing is supported to solve pre-aggregation issues.

  • The semi-structured JSON objects support the following features:

    • Storage of JSON objects is supported.

    • The JSON data of each property is stored in a separate column.

    • An experimental feature is supported. You can set the allow_experimental_object_type parameter to 1 to enable the feature.

• Optimized performance

  • The performance of INSERT operations is doubled.

  • The performance of IN queries for long tables is tripled.

• Date32

• Datetime64

• DELETE Statement

• SQL User Defined Functions

• Executable User Defined Functions

• Projections

• Semi-Structured Approach

Feature

Description

References

Parameter configuration

New feature

The config.xml file of ApsaraDB for ClickHouse clusters that are created after December 1, 2021 can be modified, and the change history can be queried.

Set parameters in the config.xml file

Feature

Description

References

Update of minor engine versions

New feature

The minor engine version of ApsaraDB for ClickHouse clusters that are purchased after December 1, 2021 can be updated. The update enhances performance, introduces new features, and fixes bugs.

Update the minor engine version

Setting of a maintenance window

New feature

The maintenance window can be set for one or more ApsaraDB for ClickHouse clusters at a time. The clusters are upgraded within the specified maintenance window to ensure the stability of the clusters.

Set a maintenance window

Feature

Description

References

Monitoring and alerting

• New feature

  The following metrics are added:

  • Number of Inactive Data Parts

  • Number of TCP connections

  • Number of running Init Query

  • Number of HTTP connections

  • Number of MaterializeMySQL

  • Number of Failed Insert Queries

  • Number of Kafka Appearances

  • Number of Failed Select Queries

  • Number of MaterializeMySQL synchronization failures

  • Number of Kafka Appearance Consumption Errors

• Optimization

  The availability of clusters and the capabilities of monitoring exceptions are improved. This improves O&M stability.

View monitoring data in the ApsaraDB for ClickHouse console

Feature

Description

References

Data backup and data restoration

New feature

ApsaraDB for ClickHouse clusters of V20.3, V20.8, and V21.8 support data backup and data restoration.

• Data backup

• Data restoration

Feature

Description

References

HTTPS

New feature

HTTPS can be used to implement encrypted data transmission.

• Enable HTTPS

• Connect to an ApsaraDB for ClickHouse cluster over HTTPS

MaterializeMySQL engine upgrade

• New feature

  The MaterializeMySQL engine supports the dirty data of the ENUM data type.

• Optimization

  • Full data synchronization is optimized. Tables are not locked during a synchronization process. This reduces the negative impact of the operation on your business.

  • DDL synchronization is optimized. Information about errors that occur during a synchronization process is written to the system table.

None

TCP keepalive

New feature:

By default, the TCP keepalive feature is enabled to ensure connection reliability.

None

Feature

Description

References

MaterializeMySQL engine upgrade

New feature

The MaterializeMySQL engine supports the DECIMAL data type.

None

Feature

Description

References

V21.8

• New feature

  • More SQL statements and JSON standards are supported.

  • The MaterializedPostgreSQL table engine and the MaterializedPostgreSQL database engine are added to allow synchronous replication of databases and tables.

  • The following functions are added: leftPad(), rightPad(), leftPadUTF8(), and rightPadUTF8().

  • The bin and unbin functions are added.

  • The FIRST keyword is added to the ADD INDEX command. This keyword allows you to add an index to the beginning of an index list.

  • The following data types are supported in the MapAdd and MapSubTract functions: MAP, UINT128, INT128, UINT256, and INT256.

  • The DISTINCT ON (columns) expression is supported.

• Optimization

  The latency of a short query that reads data from multiple table columns is reduced.

None

Feature

Description

References

Preservation of binary log files

The enable_binlog_reserved parameter can be set to enable the preservation feature for binary log files in the MaterializeMySQL engine. This feature prevents data synchronization failures that occur when binary log files are deleted. This helps improve the stability of the MaterializeMySQL engine.

MaterializeMySQL

Feature

Description

References

Cluster restart

ApsaraDB for ClickHouse allows you to restart a cluster. If an ApsaraDB for ClickHouse cluster becomes unavailable or the cluster must be restarted to fix issues, you can restart the cluster.

Restart an ApsaraDB for ClickHouse cluster

Monitoring page optimization

The layout of the monitoring page is optimized to improve user experience.

None

Support for ApsaraDB RDS for MySQL analytic instances

ApsaraDB RDS for MySQL analytic instances are supported by ApsaraDB for ClickHouse. You can synchronize data from an ApsaraDB RDS for MySQL instance to an ApsaraDB for ClickHouse cluster. This helps you analyze business data in a centralized manner.

Create an ApsaraDB for ClickHouse instance

Feature

Description

References

Data migration between ApsaraDB for ClickHouse clusters

The data migration feature is available in the ApsaraDB for ClickHouse console. You can use this feature to migrate data between ApsaraDB for ClickHouse clusters of the same engine version or from a cluster of an earlier version to a cluster of a later version.

Migrate data between ApsaraDB for ClickHouse clusters

Feature

Description

References

Multiple scale-out methods

Simple scale-out is supported. You can select one of the following scale-out methods:

• Migration scale-out

• Simple scale-out

Scale-in and scale-out

MaterializeMySQL engine upgrade

• The JSON, TIME, and YEAR data types are supported.

• Bugs that are related to keywords and table creation for incremental data are fixed.

• The following parameters are supported:

  • skip_error_count

  • alter include_tables

MaterializeMySQL

Performance optimization for horizontal scaling

clickhouse-copier is optimized to improve the efficiency of horizontal scaling.

Modify the configurations of an ApsaraDB for ClickHouse cluster

Feature

Description

References

Data backup and data restoration

The data backup and data restoration feature is available. This feature prevents data loss and improves data security.

Data backup

Feature

Description

References

Tiered storage of hot and cold data

Cold data and hot data can be automatically migrated. This reduces storage costs by more than 80% and does not degrade performance.

Tiered storage of hot data and cold data

PL2 and PL3 enhanced SSDs (ESSDs)

PL2 and PL3 ESSDs can be used to store data. This improves I/O throughput by several times.

None

Disk encryption

All data on data disks is encrypted based on block storage. If the data is leaked, unauthorized users cannot decrypt the data. This ensures data security.

Disk Encryption

Feature

Description

References

Availability of the China (Hohhot) and China (Hong Kong) regions

None

None

Log shipping optimization

Log data in Simple Log Service can be shipped to ApsaraDB for ClickHouse for real-time analytics.

Synchronize data from Log Service

Feature

Description

References

Availability of the China (Zhangjiakou), China (Chengdu), and Japan (Tokyo) regions

None

None

Account management and permission management on databases and data dictionaries

The features for managing accounts and permissions on databases and data dictionaries are available in the ApsaraDB for ClickHouse console. After you log on to the ApsaraDB for ClickHouse console, you can create accounts and change the passwords of the accounts. You can also grant new users the permissions on databases and data dictionaries to new users and modify the permissions of existing users.

Manage database accounts of an ApsaraDB for ClickHouse cluster

Change of global parameter values

The values of global parameters can be changed in the ApsaraDB for ClickHouse console. The change takes effect after your cluster is restarted.

Set parameters in the config.xml file

Feature

Description

References

Balanced data distribution during a scale-out operation

Data is evenly distributed when nodes are added to a cluster. This prevents data skew issues.

None

Query management

Query management is supported. Slow SQL queries can be identified, and running queries can be interrupted.

Monitor slow SQL queries and manage running SQL queries

New specifications

The specifications of 4 cores and 16 GB memory are added to improve the processing capability of single nodes.

Billable items

Data dictionary

Data dictionaries can be configured in the ApsaraDB for ClickHouse console. If constant data exists or tables that store business data can be split into fact tables and dimension tables, data dictionaries can be used as a substitute for the dimension tables. This allows you to query the data dictionaries to obtain dimension data and improves query efficiency.

Data dictionaries

Resource queue

Resource queues are used to resolve the following issue that may occur in open source ClickHouse: A large query that consumes a large number of resources can cause an instance to break down.

Resource queues in ApsaraDB for ClickHouse

Feature

Description

References

Integration with Quick business intelligence (BI)

ApsaraDB for ClickHouse clusters can be added as data sources in Quick BI to build BI systems.

Quick BI

Feature

Description

References

V20.3

ApsaraDB for ClickHouse V20.3.10.75 is released to fix engine bugs and enhance stability.

None

Real-time data migration or synchronization from MySQL databases

Data in MySQL databases can be migrated or synchronized to ApsaraDB for ClickHouse clusters in real time. This allows you to interact with MySQL databases and analyze business data in real time.

Import data from ApsaraDB RDS for MySQL or PolarDB for MySQL

Online log shipping and analytics

The online log shipping and analytics feature is supported. Enterprise users need to only create shipping tasks in the ApsaraDB for ClickHouse console and ship behavioral log data from Simple Log Service to ApsaraDB for ClickHouse. This allows you to use ApsaraDB for ClickHouse to analyze log data from multiple dimensions.

Synchronize data from Log Service

Feature

Description

References

Integration with Data Management (DMS)

ApsaraDB for ClickHouse is integrated with DMS. This allows you to manage databases and tables, query data, and orchestrate and schedule SQL tasks online.

None

Feature

Description

References

New specifications

The C4, C24, C64, and C104 specifications are added. HTTP ports are enabled.

Billable items