In an Alibaba Cloud E-MapReduce (EMR) ClickHouse cluster, you can configure the ClickHouse service using four types of parameters. This document describes the client, server, and extended configuration parameters.
Background
| Parameter | Details |
| client configuration | client-config |
| server configuration | server-config |
| extended configuration | server-metrika |
| user permissions | access control |
Prerequisites
You must have an E-MapReduce ClickHouse cluster. For more information, see Create a ClickHouse cluster.
Usage notes
- If a parameter can be placed directly under the yandex tag, add it directly.
- For nested parameters, use a period (.) to separate each level.
For example, on the server-users tab, you can add a new user named aliyun by setting the parameter to users.aliyun.password and providing a custom value for the password.
- Do not use XML as a parameter or value in custom configurations.
client-config
These configuration parameters generate the config.xml file used by the clickhouse-client. In the EMR console, navigate to the Configure tab of the ClickHouse service and click the client-config tab to view these parameters.
| Parameter | Description |
| user | The username for the clickhouse-client. The default value is default. |
| password | The password for the clickhouse-client user. The default value is empty. |
| prompt_by_server_display_name.production | Allows you to customize the command prompt in the clickhouse-client. The displayed prompt depends on the display_name value that is set in the server-config settings. For example, if display_name is set to default, the prompt that is defined in prompt_by_server_display_name.default is used. For information about prompt colors, see Color prompts with readline and tip_colors_and_formatting. |
| prompt_by_server_display_name.default | |
| prompt_by_server_display_name.test |
server-config
These server configuration parameters generate the config.xml file used by the clickhouse-server process. In the EMR console, navigate to the Configure tab of the ClickHouse service and click the server-config tab to view these parameters.
| Parameter | Description |
| tcp_port | The TCP port for client communication. The default value is 9000. |
| logger.count | The number of archived ClickHouse log files to retain. When the number of archived log files reaches this value, ClickHouse deletes the oldest archive. The default value is 10. |
| logger.level | The log level. The default is information. The levels from highest to lowest severity are: none (disables logging), fatal, critical, error, warning, notice, information, debug, and trace. |
| logger.size | The maximum size of a log file. When a log file reaches this size, ClickHouse archives and renames it, and then creates a new log file. The default value is 1000M. |
| distributed_ddl.path | Specifies the path in ZooKeeper for storing the DDL query queue. The default value is /clickhouse/task_queue/ddl. By default, operations such as CREATE, DROP, ALTER, and RENAME affect only the node processing the query. You can set the distributed_ddl parameters to run these queries across the cluster. This feature is available only when ZooKeeper is used. |
| default_database | The default database. The default value is default. |
| uncompressed_cache_size | The size of the cache for uncompressed blocks when the MergeTree table engine is used. The default value is 0. A value of 0 disables the cache. |
| timezone | The time zone of the server. The default value is Asia/Shanghai. |
| max_session_timeout | The maximum session timeout in seconds. The default value is 3600. |
| default_session_timeout | The default session timeout in seconds. The default value is 60. |
| max_concurrent_queries | The maximum number of concurrently processed queries. The default value is 0. |
| keep_alive_timeout | The number of seconds that ClickHouse waits for an incoming request before closing the connection. The default value is 10. |
| http_port | The port for connecting to the server over HTTP. The default value is 8123. The official ClickHouse Java Database Connectivity (JDBC) driver also uses this port to access ClickHouse. For more information, see clickhouse-jdbc. |
| listen_host | The IP addresses that the ClickHouse server listens on. You can specify IPv4 and IPv6 addresses. If you specify ::, all addresses are allowed. You can specify multiple IP addresses separated by commas, for example, 127.0.0.1,localhost. The default value is 0.0.0.0. |
| default_profile | The default profile to use. The default value is default. |
| mark_cache_size | The approximate cache size of the mark index when the MergeTree table engine is used. The default value is 5368709120, in bytes. |
| merge_tree.allow_remote_fs_zero_copy_replication |
Set this parameter to true to use the multi-replica mechanism of remote storage such as DiskHDFS for backups when you use the Replicated*MergeTree table engine. In this case, for data on replicas within a shard, only the metadata is backed up. |
| transaction.enable_public_ip |
ClickHouse server transactions require an IP address for identification. By default, a private IP address is used. Set this parameter to true to use a public IP address for identification. This requires that all nodes have public IP addresses. |
server-metrika
These extended configuration parameters generate the metrika.xml file, which is referenced by the ClickHouse server's configuration file. To view these parameters, go to the Configure tab of the ClickHouse service in the EMR console. The server-metrika tab is selected by default.
| Parameter | Description |
| clickhouse_compression | Configures data compression for tables that use MergeTree-related table engines. For more information, see Server Settings. The default value is empty. To enable ClickHouse compression, add the relevant configuration settings. |
| storage_configuration | Specifies custom disk information. |
| zookeeper_servers | Specifies the ZooKeeper information for the ClickHouse cluster. By default, this parameter is set to the ZooKeeper cluster that is created with the ClickHouse cluster. If there are multiple ZooKeeper nodes, separate them with commas, for example, emr-header-1.cluster-12345:2181,emr-worker-1.cluster-12345:2181,emr-worker-2.cluster-12345:2181. |
| quotas_default | ClickHouse allows you to configure quotas to set various resource limits. Modifying this configuration parameter updates the quota named default. To add a new quota, add custom configuration parameters. |
| clickhouse_remote_servers | Customizes the shard and replica information of the cluster. The default value is the topology generated based on the shard and replica counts specified when you created the ClickHouse cluster. Important Change the value of this parameter only when it is necessary. We recommend that you do not manually change the number of shards, the number of replicas, or the topology. Otherwise, errors may occur when you write data to or query data from a ClickHouse cluster. |