This topic describes the Frontend (FE), Backend (BE), and system parameters for StarRocks clusters on E-MapReduce (EMR), and how to modify them.
Parameters fall into two categories:
-
Dynamic parameters: Take effect immediately without restarting the service. All FE dynamic parameters can be modified online.
-
Static parameters: Take effect only after a service restart.
View current configuration
To view the current FE configuration, run the following command in a MySQL client connected to the FE:
ADMIN SHOW FRONTEND CONFIG;
The output includes the parameter name, current value, and whether the parameter is mutable (can be modified online).
Modify FE dynamic parameters
Run the following command in a MySQL client connected to the FE:
ADMIN SET FRONTEND CONFIG ("key" = "value");
FE dynamic parameters
Logging
| Parameter | Default value | Description |
|---|---|---|
qe_slow_log_ms |
5000 |
Threshold for slow query detection. Queries that take longer than this value are recorded as slow queries. Unit: milliseconds. |
Metadata and cluster management
| Parameter | Default value | Description |
|---|---|---|
catalog_try_lock_timeout_ms |
5000 |
Timeout for acquiring a catalog lock. Unit: milliseconds. |
edit_log_roll_num |
50000 |
Maximum number of image log entries written before a new log file is created. |
ignore_unknown_log_id |
FALSE |
Whether to ignore unknown log IDs. Set to TRUE to ignore them; set to FALSE to exit the FE when unknown log IDs are encountered. Unknown log IDs can appear when an FE is rolled back to an earlier version that BEs do not recognize. |
ignore_meta_check |
FALSE |
Whether to ignore the metadata gap. Set to TRUE to ignore the gap; set to FALSE to enforce the check. |
max_backend_down_time_second |
3600 |
Maximum time for a BE to rejoin the cluster after disconnecting from the FE. If the BE does not reconnect within this period, it is considered permanently down. Unit: seconds. |
drop_backend_after_decommission |
TRUE |
Whether to automatically delete a BE node after decommissioning. Set to TRUE to delete it; set to FALSE to retain it. |
Query engine
| Parameter | Default value | Description |
|---|---|---|
expr_children_limit |
10000 |
Maximum number of IN predicate elements allowed in a query. |
expr_depth_limit |
3000 |
Maximum nesting depth for expressions in a query. |
max_allowed_in_element_num_of_delete |
10000 |
Maximum number of elements in the IN predicate of a DELETE statement. |
max_layout_length_per_row |
2147483647 |
Maximum length of a single row. |
disable_cluster_feature |
TRUE |
Whether to disable the logical cluster feature. Set to TRUE to disable it; set to FALSE to enable it. |
enable_materialized_view |
TRUE |
Whether to enable materialized views. Set to TRUE to enable; set to FALSE to disable. |
enable_decimal_v3 |
TRUE |
Whether to support the DECIMAL V3 data type. Set to TRUE to enable; set to FALSE to disable. |
enable_sql_blacklist |
FALSE |
Whether to enforce the SQL query blacklist. When enabled, queries matching blacklist patterns cannot be executed. Set to TRUE to enable; set to FALSE to disable. |
dynamic_partition_check_interval_seconds |
600 |
Interval at which the FE checks for new data and automatically creates partitions. Unit: seconds. |
dynamic_partition_enable |
TRUE |
Whether to enable dynamic partitioning. Set to TRUE to enable; set to FALSE to disable. |
max_partitions_in_one_batch |
4096 |
Maximum number of partitions that can be created in a single batch. |
max_query_retry_time |
2 |
Maximum number of query retries on the FE. |
max_create_table_timeout_second |
60 |
Maximum timeout for creating a table. Unit: seconds. |
max_running_rollup_job_num_per_table |
1 |
Maximum number of concurrent Rollup jobs per table. |
max_planner_scalar_rewrite_num |
100000 |
Maximum number of times the optimizer rewrites a scalar operator. |
statistics_manager_sleep_time_sec |
600 |
Interval at which statistics tables are created. Unit: seconds. |
statistic_collect_interval_sec |
7200 |
Interval at which statistics are collected. Unit: seconds. |
statistic_update_interval_sec |
86400 |
Default interval at which statistics are updated. Unit: seconds. |
statistic_sample_collect_rows |
200000 |
Number of rows sampled during statistics collection. |
enable_statistic_collect |
TRUE |
Whether to enable statistics collection. Set to TRUE to enable; set to FALSE to disable. |
enable_local_replica_selection |
FALSE |
Whether the optimizer preferentially selects tablet replicas on BEs that share the same IP address as the FE. Set to TRUE to prefer local replicas; set to FALSE for the default selection behavior. |
max_distribution_pruner_recursion_depth |
100 |
Maximum recursion depth for partition pruning. |
Loading and unloading
| Parameter | Default value | Description |
|---|---|---|
load_straggler_wait_second |
300 |
Maximum loading lag tolerated for a BE replica before data is cloned from other replicas. Unit: seconds. |
desired_max_waiting_jobs |
100 |
Maximum number of pending jobs across all job types, including table creation, loading, and schema change jobs. |
max_running_txn_num_per_db |
100 |
Maximum number of concurrent load jobs per database. |
max_load_timeout_second |
259200 |
Maximum timeout for any load job. Unit: seconds. |
min_load_timeout_second |
1 |
Minimum timeout for any load job. Unit: seconds. |
load_parallel_instance_num |
1 |
Number of concurrent loading instances on a single BE. |
disable_hadoop_load |
FALSE |
Whether to disable loading data from Hadoop. Set to TRUE to disable; set to FALSE to enable. |
disable_load_job |
FALSE |
Whether to reject new load jobs when the cluster encounters an error. Set to TRUE to reject; set to FALSE to accept. |
db_used_data_quota_update_interval_secs |
300 |
Interval at which the FE updates data quota usage for each database. Unit: seconds. |
history_job_keep_max_second |
604800 |
Maximum retention period for historical jobs. Unit: seconds. |
label_keep_max_num |
1000 |
Maximum number of load jobs retained within the period defined by label_keep_max_second. |
label_keep_max_second |
259200 |
Retention period for load job labels. Unit: seconds. |
max_routine_load_job_num |
100 |
Maximum number of Routine Load jobs. |
max_routine_load_task_concurrent_num |
5 |
Maximum number of concurrent tasks within a single Routine Load job. |
max_routine_load_task_num_per_be |
5 |
Maximum number of concurrent Routine Load tasks on a single BE. Must be less than or equal to routine_load_thread_pool_size. |
max_routine_load_batch_size |
524288000 |
Maximum data volume loaded by a single Routine Load task. Unit: bytes. |
routine_load_task_consume_second |
3 |
Maximum time a Routine Load task spends consuming data. Unit: seconds. |
routine_load_task_timeout_second |
15 |
Timeout for a Routine Load task. Unit: seconds. |
max_tolerable_backend_down_num |
0 |
Maximum number of failed BEs allowed before Routine Load jobs can no longer be automatically recovered. |
period_of_auto_resume_min |
5 |
Interval at which the FE attempts to automatically recover Routine Load jobs. Unit: minutes. |
spark_load_default_timeout_second |
86400 |
Timeout for a Spark Load job. Unit: seconds. |
spark_home_default_dir |
STARROCKS_HOME_DIR/lib/spark2x |
Root directory of the Spark client. |
stream_load_default_timeout_second |
600 |
Default timeout for a Stream Load job. Unit: seconds. |
max_stream_load_timeout_second |
259200 |
Maximum timeout for a Stream Load job. Unit: seconds. |
insert_load_default_timeout_second |
3600 |
Timeout for an INSERT INTO load operation. Unit: seconds. |
broker_load_default_timeout_second |
14400 |
Timeout for a Broker Load job. Unit: seconds. |
min_bytes_per_broker_scanner |
67108864 |
Minimum data volume processed by a single Broker Load instance. Default: 64 MB. |
max_broker_concurrency |
100 |
Maximum number of concurrent Broker Load instances. |
export_max_bytes_per_be_per_task |
268435456 |
Maximum data volume exported from a single BE by a single unload task. Default: 256 MB. |
export_running_job_num_limit |
5 |
Maximum number of concurrent data unload jobs. |
export_task_default_timeout_second |
7200 |
Timeout for a data unload job. Default: 2 hours. Unit: seconds. |
Storage
| Parameter | Default value | Description |
|---|---|---|
enable_strict_storage_medium_check |
FALSE |
Whether the FE checks for available storage space on BEs when creating a table. Set to TRUE to enforce the check; set to FALSE to skip it. |
capacity_used_percent_high_water |
0.75 |
Upper disk usage threshold for a BE. When disk usage exceeds 75%, the FE stops sending table creation and clone jobs to that BE until usage drops below the threshold. |
storage_high_watermark_usage_percent |
85 |
Maximum allowed storage usage percentage in a BE's storage directory. |
storage_min_left_capacity_bytes |
2147483648 |
Minimum required free space in a BE's storage directory. Default: 2 GB. |
storage_flood_stage_left_capacity_bytes |
1073741824 |
Free space threshold in a BE's storage directory below which load and restore jobs are rejected. Default: 1 GB. |
storage_flood_stage_usage_percent |
95 |
Storage usage threshold above which load and restore jobs are rejected. |
catalog_trash_expire_second |
86400 |
Retention period for metadata in the recycle bin after a table or database is deleted. Data cannot be restored after this period expires. Default: 1 day. Unit: seconds. |
alter_table_timeout_second |
86400 |
Timeout for a schema change job. Default: 1 day. Unit: seconds. |
balance_load_disk_safe_threshold |
0.5 |
Safe disk usage threshold for the disk_and_tablet balancing policy. If all BE disks are below 50% usage, disk usage is considered balanced. |
balance_load_score_threshold |
0.1 |
Load imbalance threshold. For the be_load_score policy: a BE is in a low-load state if its load is more than 10% below average, and in a high-load state if its load is more than 10% above average. For the disk_and_tablet policy: if the difference between the maximum and minimum BE disk usage exceeds 10%, the cluster initiates tablet rebalancing. |
disable_balance |
true |
Whether to disable tablet balancing. Set to TRUE to disable; set to FALSE to enable. |
max_scheduling_tablets |
2000 |
Maximum number of tablets being scheduled concurrently. If this threshold is exceeded, tablet load balancing is skipped. |
max_balancing_tablets |
100 |
Maximum number of tablets being rebalanced concurrently. If this threshold is exceeded, rebalancing is skipped. |
disable_colocate_balance |
FALSE |
Whether to disable replica balancing for colocate tables. Set to TRUE to disable; set to FALSE to enable. |
recover_with_empty_tablet |
FALSE |
Whether to substitute an empty tablet replica when a replica is lost or corrupted. Set to TRUE to allow substitution (queries can still run but may return incomplete results); set to FALSE to disable substitution. |
min_clone_task_timeout_sec |
180 |
Minimum timeout for a tablet clone task. Default: 3 minutes. Unit: seconds. |
max_clone_task_timeout_sec |
7200 |
Maximum timeout for a tablet clone task. Default: 2 hours. Unit: seconds. |
tablet_create_timeout_second |
1 |
Timeout for creating a tablet. Unit: seconds. |
tablet_delete_timeout_second |
2 |
Timeout for deleting a tablet. Unit: seconds. |
tablet_repair_delay_factor_second |
60 |
Interval between tablet replica repair attempts. Unit: seconds. |
consistency_check_start_time |
23 |
Hour at which the FE starts replica consistency checks. Default: 23:00. |
consistency_check_end_time |
4 |
Hour at which the FE stops replica consistency checks. Default: 04:00. |
check_consistency_default_timeout_second |
600 |
Timeout for a replica consistency check. Unit: seconds. |
Others
| Parameter | Default value | Description |
|---|---|---|
plugin_enable |
TRUE |
Whether to enable the plug-in feature. Plug-ins can only be installed or uninstalled on the master FE node. Set to TRUE to enable; set to FALSE to disable. |
max_small_file_number |
100 |
Maximum number of small files that can be stored. |
max_small_file_size_bytes |
1048576 |
Maximum size of a single small file. Default: 1 MB. |
backup_job_default_timeout_ms |
86400000 |
Timeout for a backup job. Default: 1 day. Unit: milliseconds. |
report_queue_size |
100 |
Maximum number of pending reports in the queue. Reports include disk status, task status, and tablet information from BEs. |
FE static parameters
Static parameters take effect only after an FE restart.
Logging
| Parameter | Default value | Description |
|---|---|---|
log_roll_size_mb |
1024 |
Maximum size of a log file before a new one is created. Default: 1 GB. |
sys_log_dir |
StarRocksFe.STARROCKS_HOME_DIR/log |
Directory for system log files. |
sys_log_level |
INFO |
Minimum severity level for system log entries. Valid values: INFO, WARNING, ERROR, FATAL. |
sys_log_verbose_modules |
_(empty)_ | Modules for which verbose system logs are generated. For example, setting this to org.apache.starrocks.catalog generates logs only for the catalog module. |
sys_log_roll_interval |
DAY |
Rotation interval for system log files. |
sys_log_delete_age |
7d |
Retention period for system log files. Files older than this value are deleted. |
sys_log_roll_num |
10 |
Maximum number of system log files retained per rotation interval. |
audit_log_dir |
starrocksFe.STARROCKS_HOME_DIR/log |
Directory for audit log files. |
audit_log_roll_num |
90 |
Maximum number of audit log files retained. |
audit_log_modules |
"slow_query", "query" |
Modules for which audit logs are generated. |
audit_log_roll_interval |
DAY |
Rotation interval for audit log files. Valid values: DAY, HOUR. |
audit_log_delete_age |
30d |
Retention period for audit log files. Files older than this value are deleted. |
dump_log_dir |
STARROCKS_HOME_DIR/log |
Directory for dump log files. |
dump_log_modules |
"query" |
Modules for which dump logs are generated. |
dump_log_roll_interval |
DAY |
Rotation interval for dump log files. Files rotated daily use a yyyyMMdd suffix; files rotated hourly use a yyyyMMddHH suffix. |
dump_log_roll_num |
90 |
Maximum number of dump log files retained per rotation interval. |
dump_log_delete_age |
30d |
Retention period for dump log files. Files older than this value are deleted. |
Server
| Parameter | Default value | Description |
|---|---|---|
frontend_address |
0.0.0.0 |
IP address of the FE node. |
priority_networks |
_(empty)_ | Preferred CIDR block for servers with multiple network interfaces. For example: 10.10.**/24. |
http_port |
8030 |
HTTP server port. |
http_backlog_num |
1024 |
Backlog queue length for the HTTP server. |
cluster_name |
StarRocks Cluster |
Cluster name displayed in the web UI. |
rpc_port |
9020 |
Thrift server port on the FE. |
thrift_backlog_num |
1024 |
Backlog queue length for the Thrift server. |
thrift_server_type |
THREAD_POOL |
Service model for the Thrift server. Valid values: SIMPLE, THREADED, THREAD_POOL. |
thrift_server_max_worker_threads |
4096 |
Maximum number of worker threads for the Thrift server. |
thrift_client_timeout_ms |
0 |
Timeout for client requests. The default value 0 means requests never time out. |
brpc_idle_wait_max_time |
10000 |
Maximum idle wait time for bRPC connections. Default: 10 seconds. Unit: milliseconds. |
query_port |
9030 |
MySQL server port on the FE. |
mysql_service_nio_enabled |
FALSE |
Whether to enable asynchronous I/O for the MySQL service. Set to TRUE to enable; set to FALSE to disable. |
mysql_service_io_threads_num |
4 |
Maximum number of threads for handling MySQL I/O events. |
mysql_nio_backlog_num |
1024 |
Backlog queue length for the MySQL server. |
max_mysql_service_task_threads_num |
4096 |
Maximum number of threads for processing MySQL tasks. |
max_connection_scheduler_threads_num |
4096 |
Maximum number of threads for the connection scheduler. |
qe_max_connection |
1024 |
Maximum number of concurrent connections to the FE across all users. |
check_java_version |
TRUE |
Whether to verify that the Java runtime version is compatible with the compiled Java version. Set to TRUE to enable; set to FALSE to skip. |
Metadata and cluster management
| Parameter | Default value | Description |
|---|---|---|
meta_dir |
StarRocksFe.STARROCKS_HOME_DIR/meta |
Directory for storing FE metadata. |
heartbeat_mgr_threads_num |
8 |
Number of threads used by Heartbeat Manager to process heartbeat tasks. |
heartbeat_mgr_blocking_queue_size |
1024 |
Size of the blocking queue for heartbeat tasks. |
metadata_failure_recovery |
FALSE |
Whether to forcibly reset FE metadata. Exercise caution when enabling this parameter. |
edit_log_port |
9010 |
Port used for communication among master, follower, and observer FE nodes. |
edit_log_type |
BDB |
Type of edit log. Set this to BDB. |
bdbje_heartbeat_timeout_second |
30 |
Heartbeat timeout for the BDB JE-based FE. Unit: seconds. |
bdbje_lock_timeout_second |
1 |
Lock timeout for the BDB JE-based FE. Unit: seconds. |
max_bdbje_clock_delta_ms |
5000 |
Maximum allowed clock offset between the master FE and follower or observer FEs. Unit: milliseconds. |
txn_rollback_limit |
100 |
Maximum number of transactions that can be rolled back. |
bdbje_replica_ack_timeout_second |
10 |
Maximum time the master FE waits for ACK messages from follower FEs. Unit: seconds. |
master_sync_policy |
SYNC |
Policy for flushing logs to disk on the master FE. |
replica_sync_policy |
SYNC |
Policy for flushing logs to disk on follower FE replicas. |
meta_delay_toleration_second |
300 |
Maximum lag allowed for metadata on follower and observer FEs relative to the master FE. Unit: seconds. |
cluster_id |
-1 |
Cluster ID. FE and BE nodes with the same cluster ID belong to the same cluster. The default value -1 causes a random cluster ID to be generated when the master FE starts for the first time. |
Query engine
| Parameter | Default value | Description |
|---|---|---|
disable_colocate_join |
FALSE |
Whether to disable colocate joins. Set to FALSE to enable; set to TRUE to disable. |
enable_udf |
FALSE |
Whether to enable user-defined functions (UDFs). Set to TRUE to enable; set to FALSE to disable. |
publish_version_interval_ms |
10 |
Interval at which version publish tasks are dispatched. Unit: milliseconds. |
statistic_cache_columns |
100000 |
Maximum number of rows cached for the statistics table. |
Loading and unloading
| Parameter | Default value | Description |
|---|---|---|
async_load_task_pool_size |
10 |
Thread pool size for asynchronous load jobs. |
load_checker_interval_second |
5 |
Polling interval for load job status checks. Unit: seconds. |
transaction_clean_interval_second |
30 |
Interval at which completed transactions are cleaned up. Unit: seconds. |
label_clean_interval_second |
14400 |
Interval at which load job labels are cleaned up. Unit: seconds. |
spark_dpp_version |
1.0.0 |
Version of Spark Dynamic Partition Pruning (DPP). |
spark_resource_path |
_(empty)_ | Root directory for Spark dependency packages. |
spark_launcher_log_dir |
sys_log_dir/spark_launcher_log |
Directory for Spark launcher log files. |
yarn_client_path |
STARROCKS_HOME_DIR/lib/yarn-client/hadoop/bin/yarn |
Path to the YARN client binary. |
yarn_config_dir |
STARROCKS_HOME_DIR/lib/yarn-config |
Directory for YARN configuration files. |
export_checker_interval_second |
5 |
Polling interval for unload job scheduling. Unit: seconds. |
export_task_pool_size |
5 |
Thread pool size for unload jobs. |
Storage
| Parameter | Default value | Description |
|---|---|---|
storage_cooldown_second |
2592000 |
Interval for storage media migration from SSD to HDD. Default: 30 days. Unit: seconds. |
default_storage_medium |
HDD |
Default storage media for tables and partitions when no media is specified. Valid values: HDD, SSD. |
schedule_slot_num_per_path |
2 |
Maximum number of tablet jobs that can run concurrently in each storage directory on a BE. |
tablet_balancer_strategy |
disk_and_tablet |
Load balancing policy for tablets. Valid values: disk_and_tablet, be_load_score. |
tablet_stat_update_interval_second |
300 |
Interval at which the FE retrieves tablet statistics from each BE. Default: 5 minutes. Unit: seconds. |
Others
| Parameter | Default value | Description |
|---|---|---|
plugin_dir |
STARROCKS_HOME_DIR/plugins |
Directory for plug-in installation packages. |
small_file_dir |
STARROCKS_HOME_DIR/small_files |
Root directory for small files. |
max_agent_task_threads_num |
4096 |
Maximum number of threads in the agent task thread pool. |
authentication_ldap_simple_bind_base_dn |
"" |
Base DN for searching user authentication information in LDAP. |
authentication_ldap_simple_bind_root_dn |
"" |
Administrator DN for searching user authentication information in LDAP. |
authentication_ldap_simple_bind_root_pwd |
"" |
Password for the LDAP administrator DN. |
authentication_ldap_simple_server_host |
"" |
Hostname of the LDAP server. |
authentication_ldap_simple_server_port |
389 |
Port of the LDAP server. |
authentication_ldap_simple_user_search_attr |
uid |
LDAP attribute used to identify users. |
tmp_dir |
starrocksFe.STARROCKS_HOME_DIR/temp_ddir |
Directory for temporary files such as those generated during backup and restore. |
locale |
zh_CN.UTF-8 |
Character set encoding. |
hive_meta_load_concurrency |
4 |
Maximum number of concurrent threads for loading Hive metadata. |
hive_meta_cache_refresh_interval_s |
4096 |
Interval at which cached Hive external table metadata is refreshed. Unit: seconds. |
hive_meta_cache_ttl_s |
7200 |
Time-to-live for cached Hive external table metadata. Default: 2 hours. Unit: seconds. |
hive_meta_store_timeout_s |
86400 |
Connection timeout for the Hive Metastore. Default: 24 hours. Unit: seconds. |
es_state_sync_interval_second |
10 |
Interval at which the FE syncs Elasticsearch index metadata. Unit: seconds. |
enable_auth_check |
TRUE |
Whether to enable authentication checks. Set to TRUE to enable; set to FALSE to disable. |
enable_metric_calculator |
TRUE |
Whether to periodically collect cluster metrics. Set to TRUE to enable; set to FALSE to disable. |
BE parameters
All BE parameters are static parameters and take effect only after a BE restart.
| Parameter | Default value | Description |
|---|---|---|
be_port |
9060 |
Thrift server port on the BE, used to receive requests from FEs. |
brpc_port |
8060 |
bRPC port for viewing bRPC network statistics. |
brpc_num_threads |
-1 |
Number of bRPC worker threads. The default value -1 uses the same number as CPU cores. |
priority_networks |
_(empty)_ | Preferred CIDR block for servers with multiple network interfaces. For example: 10.10.**/24. |
heartbeat_service_port |
9050 |
Heartbeat service port on the BE, used to receive heartbeats from FEs. |
heartbeat_service_thread_count |
1 |
Number of threads for the heartbeat service. |
create_tablet_worker_count |
3 |
Number of threads for creating tablets. |
drop_tablet_worker_count |
3 |
Number of threads for deleting tablets. |
push_worker_count_normal_priority |
3 |
Number of threads for processing NORMAL-priority load jobs. |
push_worker_count_high_priority |
3 |
Number of threads for processing HIGH-priority load jobs. |
publish_version_worker_count |
2 |
Number of threads for publishing tablet versions. |
clear_transaction_task_worker_count |
1 |
Number of threads for clearing completed transactions. |
alter_tablet_worker_count |
3 |
Number of threads for schema change operations. |
clone_worker_count |
3 |
Number of threads for tablet cloning. |
storage_medium_migrate_count |
1 |
Number of threads for storage media migration, such as migrating hot data from SSD to HDD. |
check_consistency_worker_count |
1 |
Number of threads for checking tablet replica consistency. |
report_task_interval_seconds |
10 |
Interval at which the BE reports job status (table creation, deletion, loading, and schema change) to the FE. Unit: seconds. |
report_disk_state_interval_seconds |
60 |
Interval at which the BE reports disk status and data volumes to the FE. Unit: seconds. |
report_tablet_interval_seconds |
60 |
Interval at which the BE reports tablet version information to the FE. Unit: seconds. |
alter_tablet_timeout_seconds |
86400 |
Timeout for a schema change job on the BE. Unit: seconds. |
sys_log_dir |
${DORIS_HOME}/log |
Directory for BE system logs. |
user_function_dir |
${DORIS_HOME}/lib/udf |
Directory for UDF files. |
sys_log_level |
INFO |
Minimum severity level for BE system log entries. Valid values: INFO, WARNING, ERROR, FATAL. |
sys_log_roll_mode |
SIZE-MB-1024 |
Maximum log file size before rotation. Default: 1 GB. |
sys_log_roll_num |
10 |
Maximum number of system log files retained. |
sys_log_verbose_modules |
_(empty)_ | Modules for which verbose system logs are generated. For example, setting this to OLAP generates logs only for the OLAP module. |
sys_log_verbose_level |
10 |
Verbosity level for logs prefixed with VLOG in the source code. |
log_buffer_level |
_(empty)_ | Log flush policy. By default, logs are buffered in memory. |
num_threads_per_core |
3 |
Number of threads started per CPU core. |
compress_rowbatches |
TRUE |
Whether to compress row batches in bRPC calls between BEs during query processing. Set to TRUE to enable; set to FALSE to disable. |
serialize_batch |
FALSE |
Whether to serialize row batches in bRPC calls between BEs during query processing. Set to TRUE to enable; set to FALSE to disable. |
status_report_interval |
5 |
Interval at which the BE reports query profiles to the FE. Unit: seconds. |
doris_scanner_thread_pool_thread_num |
48 |
Number of threads in the storage engine's scan thread pool. |
doris_scanner_thread_pool_queue_size |
102400 |
Maximum number of scan tasks queued in the storage engine. |
doris_scan_range_row_count |
524288 |
Granularity at which the storage engine splits query scan ranges. |
doris_scanner_queue_size |
1024 |
Maximum number of scan tasks the storage engine supports. |
doris_scanner_row_num |
16384 |
Maximum number of data rows returned by each scan thread per call. |
doris_max_scan_key_num |
1024 |
Maximum number of scan keys that can be split. |
column_dictionary_key_ratio_threshold |
0 |
Ratio threshold for the STRING type below which dictionary compression is applied. |
column_dictionary_key_size_threshold |
0 |
Size threshold for a column below which dictionary compression is applied. |
memory_limitation_per_thread_for_schema_change |
2 |
Maximum memory per thread for schema change jobs. Unit: GB. |
file_descriptor_cache_clean_interval |
3600 |
Interval at which unused cached file descriptors are evicted. Unit: seconds. |
disk_stat_monitor_interval |
5 |
Interval at which disk status is checked. Unit: seconds. |
unused_rowset_monitor_interval |
30 |
Interval at which expired rowsets are cleared. Unit: seconds. |
storage_root_path |
_(empty)_ | Root directory for data storage. |
max_tablet_num_per_shard |
1024 |
Maximum number of tablets per shard within each storage directory. |
pending_data_expire_time_sec |
1800 |
Maximum retention period for pending data in the storage engine. Unit: seconds. |
inc_rowset_expired_sec |
1800 |
Retention period for loaded data in incremental cloning scenarios. Unit: seconds. |
max_garbage_sweep_interval |
3600 |
Maximum interval for disk garbage collection. Unit: seconds. |
min_garbage_sweep_interval |
180 |
Minimum interval for disk garbage collection. Unit: seconds. |
snapshot_expire_time_sec |
172800 |
Retention period for snapshots before they are deleted. Default: 48 hours. Unit: seconds. |
trash_file_expire_time_sec |
259200 |
Retention period for files in the recycle bin before they are deleted. Default: 72 hours. Unit: seconds. |
file_descriptor_cache_capacity |
16384 |
Maximum number of cached file descriptors. |
min_file_descriptor_number |
60000 |
Minimum number of file descriptors for the BE process. |
index_stream_cache_capacity |
10737418240 |
Cache capacity for Bloom filter, Min, and Max statistical information. |
storage_page_cache_limit |
0 |
Maximum capacity of the page cache. |
disable_storage_page_cache |
TRUE |
Whether to disable the page cache. Set to TRUE to disable; set to FALSE to enable. |
base_compaction_check_interval_seconds |
60 |
Thread polling interval for base compaction. Unit: seconds. |
base_compaction_num_threads_per_disk |
1 |
Number of threads for base compaction per disk. |
base_cumulative_delta_ratio |
0.3 |
Ratio of cumulative file size to base file size that triggers a base compaction. |
base_compaction_interval_seconds_since_last_operation |
86400 |
Minimum interval since the last base compaction that triggers another base compaction. Unit: seconds. |
cumulative_compaction_check_interval_seconds |
1 |
Thread polling interval for cumulative compaction. Unit: seconds. |
min_cumulative_compaction_num_singleton_deltas |
5 |
Minimum number of singleton files that triggers a cumulative compaction. |
max_cumulative_compaction_num_singleton_deltas |
1000 |
Maximum number of singleton files that triggers a cumulative compaction. |
cumulative_compaction_num_threads_per_disk |
1 |
Number of threads for cumulative compaction per disk. |
min_compaction_failure_interval_sec |
120 |
Minimum interval before a failed tablet compaction can be rescheduled. Unit: seconds. |
max_compaction_concurrency |
-1 |
Maximum concurrency for base compactions and cumulative compactions combined. The default value -1 means no limit. |
webserver_port |
8040 |
HTTP server port on the BE. |
webserver_num_workers |
48 |
Number of worker threads for the BE HTTP server. |
periodic_counter_update_period_ms |
500 |
Interval at which Counter statistics are collected. Unit: milliseconds. |
load_data_reserve_hours |
4 |
Retention period for files generated by small-scale load jobs. Unit: hours. |
load_error_log_reserve_hours |
48 |
Retention period for load error log files. Unit: hours. |
number_tablet_writer_threads |
16 |
Number of threads for Stream Load. |
streaming_load_max_mb |
10240 |
Maximum file size for a single Stream Load. Unit: MB. |
streaming_load_rpc_max_alive_time_sec |
1200 |
RPC timeout for Stream Load. Unit: seconds. |
fragment_pool_thread_num |
64 |
Number of initial threads for query execution. Additional threads are created dynamically as needed. |
fragment_pool_queue_size |
2048 |
Maximum number of queries that can be queued on a single BE node. |
enable_partitioned_aggregation |
TRUE |
Whether to enable partition-based aggregation. Set to TRUE to enable; set to FALSE to disable. |
enable_token_check |
TRUE |
Whether to enable token-based authentication checks. Set to TRUE to enable; set to FALSE to disable. |
load_process_max_memory_limit_bytes |
107374182400 |
Maximum memory for all load threads on a BE node. Default: 100 GB. |
load_process_max_memory_limit_percent |
30 |
Maximum percentage of total memory used by all load threads on a BE node. |
sync_tablet_meta |
FALSE |
Whether to enable synchronous writes for tablet metadata. Set to TRUE to enable; set to FALSE to disable. |
thrift_rpc_timeout_ms |
5000 |
Timeout for Thrift RPC calls. Unit: milliseconds. |
txn_commit_rpc_timeout_ms |
10000 |
Timeout for transaction commit RPC calls. Unit: milliseconds. |
routine_load_thread_pool_size |
10 |
Thread pool size for Routine Load. Must be greater than or equal to max_routine_load_task_num_per_be. |
tablet_meta_checkpoint_min_new_rowsets_num |
10 |
Minimum number of new rowsets since the last tablet meta checkpoint before triggering a new checkpoint. |
tablet_meta_checkpoint_min_interval_secs |
600 |
Thread polling interval for generating tablet meta checkpoints. Unit: seconds. |
brpc_max_body_size |
209715200 |
Maximum body size for bRPC messages. Default: 200 MB. |
max_runnings_transactions |
2000 |
Maximum number of concurrent transactions supported by the storage engine. |
tablet_map_shard_size |
32 |
Size of the tablet map shard. |
enable_bitmap_union_disk_format_with_set |
FALSE |
Whether to enable the new on-disk storage format for the BITMAP type, which improves bitmap_union performance. Set to TRUE to enable; set to FALSE to disable. |