E-MapReduce:Configuration items of backend nodes

alter_tablet_worker_count

Number of threads for schema change operations.

generate_compaction_tasks_min_interval_ms

Minimum interval between two consecutive compaction task generations.

enable_vectorized_compaction

Whether to enable vectorized compaction.

base_compaction_interval_seconds_since_last_operation

Elapsed time since the last base compaction. This is one of the conditions that trigger base compaction.

base_compaction_num_cumulative_deltas

Maximum number of cumulative files before base compaction is triggered. This is one of the trigger conditions for base compaction.

base_compaction_write_mbytes_per_sec

Maximum disk write speed for a single base compaction task.

base_cumulative_delta_ratio

Ratio of cumulative file sizes to base file sizes. When the ratio reaches this threshold, base compaction is triggered. This is one of the trigger conditions for base compaction.

base_compaction_trace_threshold

Time threshold for logging base compaction trace information. When a base compaction task runs longer than this value, its trace is written to the log. Set to a higher value to reduce log volume in busy environments.

The trace log entry looks like:

W0610 11:26:33.804431 56452 storage_engine.cpp:552] Trace:
0610 11:23:03.727535 (+     0us) storage_engine.cpp:554] start to perform base compaction
0610 11:23:03.728961 (+  1426us) storage_engine.cpp:560] found best tablet 546859
0610 11:23:03.728963 (+     2us) base_compaction.cpp:40] got base compaction lock
0610 11:23:03.729029 (+    66us) base_compaction.cpp:44] rowsets picked
0610 11:24:51.784439 (+108055410us) compaction.cpp:46] got concurrency lock and start to do compaction
0610 11:24:51.784818 (+   379us) compaction.cpp:74] prepare finished
0610 11:26:33.359265 (+101574447us) compaction.cpp:87] merge rowsets finished
0610 11:26:33.484481 (+125216us) compaction.cpp:102] output rowset built
0610 11:26:33.484482 (+     1us) compaction.cpp:106] check correctness finished
0610 11:26:33.513197 (+ 28715us) compaction.cpp:110] modify rowsets finished
0610 11:26:33.513300 (+   103us) base_compaction.cpp:49] compaction finished
0610 11:26:33.513441 (+   141us) base_compaction.cpp:56] unused rowsets have been moved to GC queue
Metrics: {"filtered_rows":0,"input_row_num":3346807,"input_rowsets_count":42,"input_rowsets_data_size":1256413170,"input_segments_num":44,"merge_rowsets_latency_us":101574444,"merged_rows":0,"output_row_num":3346807,"output_rowset_data_size":1228439659,"output_segments_num":6}

disable_auto_compaction

Whether to disable automatic compaction. Disable only in debugging or test environments where you want to trigger compaction manually.

disable_compaction_trace_log

Whether to disable compaction trace logging. When set to true, the cumulative_compaction_trace_threshold and base_compaction_trace_threshold items have no effect.

check_auto_compaction_interval_seconds

Interval at which the BE checks whether automatic compaction is disabled. Takes effect only when disable_auto_compaction is true.

cumulative_compaction_budgeted_bytes

Maximum size of a Singleton file. This is one of the conditions that trigger base compaction.

cumulative_compaction_trace_threshold

Time threshold for logging cumulative compaction trace information. Has no effect when disable_compaction_trace_log is true.

cumulative_compaction_rounds_for_each_base_compaction_round

Number of cumulative compaction rounds run before each base compaction round.

cumulative_compaction_policy

Merge policy for cumulative compaction. Valid values:

• ordinary — the initial policy. After cumulative compaction, the rowset immediately enters the base compaction stage.

• size_based — an optimized policy. Rowsets are merged only when they are at the same order of magnitude in disk size. A rowset is promoted to base compaction only after it meets the promotion threshold. Reduces write amplification for large numbers of small-batch imports.

cumulative_size_based_promotion_size_mbytes

When cumulative_compaction_policy is size_based, a cumulative compaction output rowset is promoted to base compaction when its total disk size exceeds this threshold. Keep this below 2048 MB to avoid version backlogs caused by long-running cumulative compaction.

cumulative_size_based_promotion_ratio

When cumulative_compaction_policy is size_based, a cumulative compaction output rowset is promoted to base compaction when its total disk size exceeds this fraction of the base rowset size. Values between 0.02 and 0.1 work well for most deployments.

cumulative_size_based_promotion_min_size_mbytes

When cumulative_compaction_policy is size_based, a cumulative compaction output rowset smaller than this threshold stays in the cumulative compaction stage and is not promoted to base compaction. Values above 512 MB may prevent base compaction from running.

cumulative_size_based_compaction_lower_size_mbytes

When cumulative_compaction_policy is size_based, rowsets larger than this threshold are split and merged using a level policy; smaller rowsets are merged directly. Values above 128 MB may cause excessive write amplification during cumulative compaction.

compaction_tablet_compaction_score_factor

Weight of the compaction score in the tablet scoring formula used to select a tablet for compaction:

tablet_score = compaction_tablet_scan_frequency_factor × tablet_scan_frequency
             + compaction_tablet_compaction_score_factor × compaction_score

compaction_tablet_scan_frequency_factor

Weight of scan frequency in the tablet scoring formula. Increase this value to prioritize compacting tablets that are scanned frequently.

compaction_task_num_per_disk

Number of compaction tasks run in parallel per hard disk drive (HDD).

compaction_task_num_per_fast_disk

Number of compaction tasks run in parallel per solid-state drive (SSD).

max_base_compaction_threads

Maximum number of threads in the base compaction thread pool.

max_cumu_compaction_threads

Maximum number of threads in the cumulative compaction thread pool.

min_compaction_threads

Minimum number of threads in the compaction thread pool.

min_compaction_failure_interval_sec

After a tablet fails to merge during cumulative compaction, the tablet must wait at least this long before it can be selected again.

total_permits_for_compaction_score

Upper limit on the total permits held by all compaction tasks. Controls memory consumption for compaction. Increase this value to allow more concurrent compaction work; decrease it to reduce memory pressure.

row_step_for_compaction_merge_log

Number of merged rows between log entries during compaction. Set to 0 to suppress compaction merge logs.

enable_quick_compaction

Whether to enable quick compaction. In workloads that import small batches frequently, quick compaction merges imported versions promptly to prevent the -235 error. "Small" is measured by row count, controlled by quick_compaction_max_rows.

quick_compaction_max_rows

Maximum row count for an import to be considered small and eligible for quick compaction. Requires enable_quick_compaction to be true.

quick_compaction_batch_size

Number of small imports that trigger a quick compaction round.

quick_compaction_min_rowsets

Minimum number of rowsets that must be selected for quick compaction to actually run.

doris_max_scan_key_num

Maximum number of scan key ranges a scan node can generate from query conditions. A higher value increases scanning parallelism but may raise scheduling overhead and slow queries in high-concurrency scenarios. An empirical starting point is 50. In high-concurrency deployments, reduce this value if increased parallelism doesn't help.

doris_scan_range_row_count

Number of rows per ScanRange when the BE splits a scanning range into multiple ScanRanges. Controls how long a single OlapScanner holds an I/O thread.

doris_scanner_queue_size

Length of the RowBatch buffer queue between TransferThread and OlapScanner. Doris scans data asynchronously: scanned RowBatches from OlapScanner wait in this queue until TransferThread picks them up.

doris_scanner_row_num

Maximum rows each scanning thread returns per execution.

doris_scanner_thread_pool_queue_size

Queue length of the scanner thread pool. When submitted tasks exceed this length, new submissions block until a slot opens.

doris_scanner_thread_pool_thread_num

Number of threads in the scanner thread pool. Determines scanning parallelism. Increase to handle more concurrent scans; too high a value increases scheduling overhead.

doris_scan_range_max_mb

Maximum amount of data read by each OlapScanner per scan.

doris_max_pushdown_conjuncts_return_rate

During a hash join, the BE pushes join conditions to OlapScanner using dynamic partition pruning. After 32768 rows are scanned, the BE checks the filter rate. If the filter rate falls below this threshold (as a percentage), the BE stops using the dynamic partition clipping condition for filtering.

max_pushdown_conditions_per_column

Maximum number of filter conditions that can be pushed down to the storage engine for a single column. Applies mainly to IN predicates (WHERE colA IN (v1, v2, v3, ...)). Increasing this value lets more IN predicate conditions reach the storage engine index, potentially reducing data scanned. However, too many conditions can increase random reads. This item can also be set at the session level.

fragment_pool_queue_size

Maximum number of concurrent query requests on a single node.

fragment_pool_thread_num_min

Minimum number of threads in the fragment execution thread pool.

fragment_pool_thread_num_max

Maximum number of threads in the fragment execution thread pool. The pool starts with fragment_pool_thread_num_min threads and grows dynamically up to this limit.

enable_partitioned_aggregation

Whether the BE uses PartitionAggregateNode for aggregation. When false, AggregationNode is used instead. Keep the default unless directed otherwise.

enable_prefetch

Whether to prefetch HashBuckets when using PartitionedHashTable for aggregation and join operations. Keep true for better performance.

enable_quadratic_probing

Whether to use quadratic probing to resolve hash conflicts in PartitionedHashTable. When false, linear probing is used. For more information, see Quadratic probing.

result_buffer_cancelled_interval_time

Interval at which result buffers are released after cancellation.

exchg_node_buffer_size_bytes

Size of the buffer queue in ExchangeNode. When the sender sends more data than the buffer can hold, subsequent sends block until space is freed.

mem_limit

Maximum percentage of server memory the BE can use. Prevents the BE from starving other processes on the machine. Values above 100% are capped at 100%.

buffer_pool_limit

Maximum proportion of allocatable memory in the buffer pool. The buffer pool manages memory for all concurrent queries using buffer pages and disk spilling. Currently applies to AggregationNode and ExchangeNode.

buffer_pool_clean_pages_limit

Maximum number of clean pages the buffer pool retains.

chunk_reserved_bytes_limit

Reserved bytes limit for the Chunk Allocator, expressed as a percentage of mem_limit. The value must be a positive multiple of 2. If the value exceeds physical memory, the physical memory size is used. Increasing this value improves performance but reduces memory available to other modules.

memory_limitation_per_thread_for_schema_change

Maximum memory a single schema change task can use.

memory_maintenance_sleep_time_s

Sleep interval between memory maintenance iterations.

memory_max_alignment

Maximum memory alignment size.

disable_mem_pools

Whether to disable the memory cache pool.

madvise_huge_pages

Whether to use Linux huge pages for memory allocation.

mmap_buffers

Whether to use mmap for memory allocation.

use_mmap_allocate_chunk

Whether to use mmap to allocate chunks. When enabled, increase vm.max_map_count (default 65530) to avoid running out of memory maps. Run the following command as root:

sysctl -w vm.max_map_count=262144
# or
echo 262144 > /proc/sys/vm/max_map_count

Also set chunk_reserved_bytes_limit to a large value to maintain performance.

load_process_max_memory_limit_bytes

Maximum memory all import threads on a single node can use. Set this to a large value to maintain load performance during Doris upgrades.

load_process_max_memory_limit_percent

Maximum percentage of server memory all import threads on a single node can use. Set this to a large value to maintain load performance during Doris upgrades.

tc_free_memory_rate

Proportion of available memory. Valid values: 0 to 100.

tc_max_total_thread_cache_bytes

Total thread cache size in TCMalloc. The limit is not strictly enforced; actual usage may exceed it. In high-stress scenarios where many threads compete for TCMalloc locks (visible as SpinLock-related stack frames), increase this value to reduce contention. For more information, see TCMalloc: Thread-Caching Malloc.

tc_use_memory_min

Minimum memory TCMalloc retains. TCMalloc does not return memory to the OS if its total usage is below this threshold.

enable_tcmalloc_hook

Whether to enable the TCMalloc hook used to track thread-local MemTracker statistics.

mem_tracker_consume_min_size_bytes

Minimum size of a TCMalloc hook operation before MemTracker is updated. Smaller allocations are accumulated to avoid frequent MemTracker calls. Reducing this value increases MemTracker accuracy but raises call frequency. Increasing it reduces call frequency but introduces inaccuracy. The maximum theoretical error is mem_tracker_consume_min_size_bytes × number of BE threads.

max_free_io_buffers

Maximum number of I/O buffers IoMgr reserves. Each buffer is between 1024 bytes and 8 MB; the total size of all reserved buffers can reach up to 2 GB.

read_size

Size of data read per I/O operation sent to the OS. At 8 MB, random and sequential I/O performance are similar, balancing latency against disk utilization.

min_buffer_size

Minimum read buffer size.

storage_root_path

Directories where the BE stores data. Separate multiple directories with semicolons (;). Optionally specify the storage medium and capacity limit for each directory.

Two syntax styles are supported:

Style 1 — suffix notation:

storage_root_path=/home/disk1/doris.HDD;/home/disk2/doris.SSD;/home/disk3/doris

• .HDD — hard disk drive (HDD)

• .SSD — solid-state drive (SSD)

• No suffix — treated as HDD

Style 2 — key-value notation:

storage_root_path=/home/disk1/doris,medium:hdd;/home/disk2/doris,medium:ssd

If all disks are the same type, specify directories without any medium suffix and update the default storage medium setting in the frontend node (FE) configuration.

default_rowset_type

Default storage format for this BE. Valid values: ALPHA, BETA. This setting applies in two cases:

• When the storage_format parameter of a table is set to Default.

• When the BE performs compaction.

disable_storage_page_cache

Whether to disable page cache for index caching. Takes effect only when default_rowset_type is BETA.

storage_page_cache_limit

Maximum size of the storage page cache.

storage_page_cache_shard_size

Shard size of StoragePageCache. Valid values: 2^n (n = 0, 1, 2, ...). Set this close to the number of CPU cores on the BE to reduce lock contention in StoragePageCache.

index_page_cache_percentage

Percentage of total page cache allocated to index pages. Valid values: 0 to 100.

index_stream_cache_capacity

Capacity of the cache for index statistics, including bloom filter, Min, and Max information.

storage_flood_stage_usage_percent

Maximum percentage of disk space a data directory can use. When this threshold and storage_flood_stage_left_capacity_bytes are both reached, writes to the directory are rejected.

storage_flood_stage_left_capacity_bytes

Minimum free space a data directory must have. Works together with storage_flood_stage_usage_percent to enforce disk write limits.

segment_cache_capacity

Maximum number of segments in Segment Cache. Increasing this value caches more segments and reduces I/O; decreasing it frees memory. Adjust based on actual access patterns.

remote_storage_read_buffer_mb

Cache size when reading files from Hadoop Distributed File System (HDFS) or Object Storage Service (OSS). Increasing this value reduces remote read calls but uses more memory.

sync_tablet_meta

Whether the storage engine calls sync to flush tablet metadata to disk after each write.

storage_strict_check_incompatible_old_format

Whether to use strict verification when reading data in a format from a previous version that is incompatible with the current version. When true, the BE exits with a fatal log entry if it encounters an incompatible format. When false, a warning is logged and the BE continues.

pending_data_expire_time_sec

Maximum time the storage engine retains unvalidated (pending) data.

write_buffer_size

Size of the in-memory write buffer before it is flushed to disk during data import. Too small a value generates many small files on the BE. Too large a value can cause RPC timeouts. If you increase this value, also increase tablet_writer_open_rpc_timeout_sec.

max_segment_num_per_rowset

Maximum number of segments in a newly generated rowset during data import. When exceeded, the import fails with a -238 error. Too many segments in a rowset can cause high memory usage during compaction and out-of-memory (OOM) errors.

string_type_length_soft_limit_bytes

Soft maximum length for string data. This limit is not strictly enforced.

default_num_rows_per_column_file_block

Number of rows per RowBlock.

zone_map_row_num_threshold

Pages with fewer rows than this threshold skip zone map creation, reducing data expansion.

column_dictionary_key_ratio_threshold

String value ratio threshold for dictionary compression. When the ratio of distinct string values is below this threshold, dictionary compression is applied.

column_dictionary_key_size_threshold

Column size threshold for dictionary compression. When the column size is below this threshold, dictionary compression is applied.

tablet_map_shard_size

Shard size of the tablet map lock. Valid values: 2^n (n = 0, 1, 2, 3, 4). Increase to reduce lock contention when managing many tablets.

tablet_meta_checkpoint_min_interval_secs

Polling interval for TabletMeta Checkpoint threads.

tablet_meta_checkpoint_min_new_rowsets_num

Minimum number of new rowsets required to trigger a TabletMeta Checkpoint.

tablet_scan_frequency_time_node_interval_second

Interval at which query_scan_count is sampled to calculate tablet scan frequency. Used together with compaction_tablet_scan_frequency_factor.

tablet_stat_cache_update_interval_second

Interval at which tablet status caches are refreshed.

tablet_rowset_stale_sweep_time_sec

Expiration time for merged rowset versions. When the time since the last rowset was created in a merged version directory exceeds this value, the directory is cleaned up and the merged rowsets are deleted. Reduce this value when disk space is tight. If set below 5 minutes (300 seconds), frontend queries against merged versions may fail with a -230 error.

max_tablet_version_num

Maximum number of versions per tablet. Prevents version accumulation caused by frequent imports or delayed compaction. Import tasks are rejected when this limit is reached.

max_tablet_num_per_shard

Maximum number of tablets per shard directory. Controls the tablet directory layout to prevent any single directory from containing too many subdirectories.

check_consistency_worker_count

Number of worker threads for calculating tablet checksums.

create_tablet_worker_count

Number of worker threads for creating tablets.

drop_tablet_worker_count

Number of threads for deleting tablets.

number_tablet_writer_threads

Number of threads for writing to tablets.

tablet_writer_open_rpc_timeout_sec

RPC timeout for opening a tablet writer on a remote BE. This timeout also applies when sending a batch (1024 rows) during fast imports. If you see send batch fail timeout errors, increase this value. When you increase write_buffer_size, also increase this timeout accordingly.

tablet_writer_ignore_eovercrowded

Whether to ignore the [E1011]The server is overcrowded bRPC error during data writes. Use this as a temporary workaround when you cannot immediately adjust brpc_socket_max_unwritten_bytes (which requires a restart). This setting only suppresses the error during writes; other RPC requests still enforce the overcrowded check.

ignore_broken_disk

Behavior when a directory specified by storage_root_path is missing or inaccessible at startup:

• true — the directory is skipped. If other directories are available, the BE starts normally.

• false — the BE stops.

ignore_load_tablet_failure

Whether to ignore tablet header metadata load failures at startup. By default, if any tablet fails to load in a storage directory, the BE stops and logs:

load tablets from header failed, failed tablets size: xxx, path=xxx

To resolve the failure without restarting, either:

• Use meta_tool to delete the tablets whose metadata cannot be loaded (when other replicas are healthy).

• Set this item to true to skip the failed tablets and start the BE.

ignore_rowset_stale_unconsistent_delete

Whether to delete outdated merged rowsets that cannot form a consistent version path. Outdated rowsets are normally deleted after 30 minutes. When false, the BE exits after reporting the inconsistency. When true, the BE logs the error and continues. Ignoring this error typically does not affect queries, but may cause a -230 error when the merged version is published from the frontend node.

inc_rowset_expired_sec

Duration for which activated incremental rowsets are retained in the storage engine during incremental cloning.

path_gc_check

Whether to enable recycling scan data thread checks.

path_gc_check_interval_second

Interval between recycling scan data thread checks.

path_gc_check_step

Step size per garbage collection check iteration.

path_gc_check_step_interval_ms

Interval between steps within a single garbage collection check.

path_scan_interval_second

Interval between path scans performed before garbage collection checks.

max_garbage_sweep_interval

Maximum interval between disk garbage collection sweeps.

min_garbage_sweep_interval

Minimum interval between disk garbage collection sweeps.

unused_rowset_monitor_interval

Interval at which expired rowsets are scanned and cleaned.

trash_file_expire_time_sec

Interval at which the recycle bin is cleaned. This limit may not be enforced when disk space is critically low.

snapshot_expire_time_sec

Interval at which snapshots are cleaned.

push_write_mbytes_per_sec

Maximum import speed. Applies to all import methods.

streaming_load_max_mb

Maximum size of CSV data in a single Stream Load. Stream Load is suited for a few GB of data and is not designed for very large datasets.

streaming_load_json_max_mb

Maximum size of JSON data in a single Stream Load. JSON data cannot be split, so all data must fit in memory before parsing.

streaming_load_rpc_max_alive_time_sec

Lifetime of a TabletsChannel. Channels that receive no data within this period are deleted.

max_send_batch_parallelism_per_job

Maximum parallelism for sending data from OlapTableSink during batch processing. If send_batch_parallelism is set higher, it is overridden by this value.

send_batch_thread_pool_thread_num

Number of threads in the SendBatch thread pool.

send_batch_thread_pool_queue_size

Queue length of the SendBatch thread pool. New submissions block when the queue is full.

high_priority_flush_thread_num_per_store

Number of flush threads reserved for HIGH priority import tasks in each storage directory.

load_task_high_priority_threshold_second

Import tasks with a timeout shorter than this value are classified as HIGH priority and use a separate flush thread pool.

flush_thread_num_per_store

Number of threads per store for flushing in-memory tables to disk.

load_data_reserve_hours

Retention period for mini load data files. Files are deleted after this duration.

load_error_log_reserve_hours

Retention period for load error logs.

min_load_rpc_timeout_ms

Minimum RPC timeout for each RPC call within a load job.

compress_rowbatches

Whether to compress RowBatches using Snappy compression during serialization.

serialize_batch

Whether to serialize RowBatches for RPCs between backend nodes.

row_nums_check

Whether to validate row counts for BE operations, CE node operations, and schema changes.

etl_thread_pool_queue_size

Queue size of the extract, transform, and load (ETL) thread pool.

push_worker_count_high_priority

Number of import threads for HIGH priority tasks.

push_worker_count_normal_priority

Number of import threads for NORMAL priority tasks.

publish_version_worker_count

Number of threads for publishing new versions.

clear_transaction_task_worker_count

Number of threads for cleaning up transactions.

delete_worker_count

Number of threads for data deletion tasks.

max_runnings_transactions_per_txn_map

Maximum number of concurrent transactions per txn_partition_map in the transaction manager. Prevents the transaction manager from accumulating too many transactions.

routine_load_thread_pool_size

Thread pool size for Routine Load tasks. Must be greater than max_concurrent_task_num_per_be (default: 5).

routine_load_consumer_pool_size

Number of cached data consumers for Routine Load.

max_consumer_num_per_group

Maximum number of consumers per data consumer group, used for Routine Load.

kafka_api_version_request

Set to false when the Kafka version is lower than 0.10.0.0.

kafka_broker_version_fallback

Fallback Kafka broker version when kafka_api_version_request is false. Valid values: 0.9.0.x, 0.8.x.y.

be_port

Port of the Thrift server on the BE. Receives requests from the frontend node.

be_service_threads

Number of execution threads in the Thrift server on the BE. Determines how many frontend requests can be processed concurrently.

heartbeat_service_port

Thrift heartbeat service port on the BE. Receives heartbeats from the frontend node.

heartbeat_service_thread_count

Number of threads running the heartbeat service on the BE. Keep the default value.

webserver_port

HTTP server port on the BE. Used for the web UI and /varz endpoint.

webserver_num_workers

Number of worker threads in the web server.

brpc_port

bRPC port for communication between backend nodes.

brpc_num_threads

Number of bRPC worker threads. Increase this value for higher queries per second (QPS). For guidance, see incubator-brpc benchmark.

brpc_max_body_size

Maximum body size for bRPC messages, mapped to the max_body_size bRPC parameter. If queries fail with body_size is too large in the BE log — typically with queries that use multi distinct, no GROUP BY, and over 1 TB of data — increase this value.

brpc_socket_max_unwritten_bytes

Maximum unwritten bytes per bRPC socket, mapped to the socket_max_unwritten_bytes bRPC parameter. If queries fail with The server is overcrowded — typically when sending large bitmap values — increase this value. Note: this item cannot be changed dynamically; use tablet_writer_ignore_eovercrowded as a temporary workaround.

transfer_large_data_by_brpc

When Tuple/Block data exceeds 1.8 GB, whether to serialize the protobuf request and send the data as an HTTP bRPC attachment. When the protobuf request exceeds 2 GB, the error Bad request, error_text=[E1003]Fail to compress request occurs. This setting prevents that error by routing large payloads through HTTP bRPC.

auto_refresh_brpc_channel

Whether to verify bRPC connection availability via a handshake RPC when obtaining a connection. If the connection is unavailable, a new one is established.

thrift_client_retry_interval_ms

Retry interval for the BE Thrift client when the frontend node's Thrift server is overwhelmed (avalanche scenario).

thrift_connect_timeout_seconds

Default connection timeout for the Thrift client.

thrift_rpc_timeout_ms

Default Thrift RPC timeout.

thrift_server_type_of_fe

Thrift service model used by the frontend node. Must match the thrift_server_type configuration on the frontend node (case-insensitive). Valid values:

• THREADED — non-blocking I/O model.

• THREAD_POOL — blocking I/O model.

txn_commit_rpc_timeout_ms

RPC timeout for transaction commit.

txn_map_shard_size

Shard size of txn_map_lock. Valid values: 2^n (n = 0, 1, 2, 3, 4). Increase to improve transaction management throughput.

txn_shard_size

Shard size of txn_lock. Valid values: 2^n (n = 0, 1, 2, 3, 4). Increase to improve transaction management throughput.

port

Port used for BE self-test.

num_cores

Number of CPU cores the BE can use. 0 reads the count from /proc/cpuinfo.

num_disks

Number of disks on the machine. 0 applies the OS default.

num_threads_per_core

Number of threads per CPU core. Values two to three times the core count keep cores busy without excessive scheduling jitter.

num_threads_per_disk

Maximum number of threads per disk, which also determines the maximum queue depth per disk.

sleep_one_second

Global sleep interval for BE threads. This is a global variable and cannot be modified at runtime.

priority_networks

CIDR blocks used to select the correct IP address on multi-homed servers. Separate multiple blocks with semicolons (;). At most one IP address should match. If no address matches the specified blocks, one is chosen at random.

priority_queue_remaining_tasks_increased_frequency

Frequency at which task priority is adjusted in BlockingPriorityQueue.

make_snapshot_worker_count

Number of threads for creating snapshots.

release_snapshot_worker_count

Number of threads for releasing snapshots.

clone_worker_count

Number of threads for cloning tasks.

storage_medium_migrate_count

Number of threads for storage medium migration (cloning between HDD and SSD).

download_worker_count

Number of download threads.

upload_worker_count

Maximum number of upload threads.

download_low_speed_limit_kbps

Minimum acceptable download speed. Downloads that fall below this speed for longer than download_low_speed_time are aborted.

download_low_speed_time

Maximum duration a download can stay below download_low_speed_limit_kbps before being aborted.

max_download_speed_kbps

Maximum download speed.

sys_log_dir

Directory where BE log files are stored.

sys_log_level

Log level. From lowest to highest severity: INFO, WARNING, ERROR, FATAL.

sys_log_roll_mode

Log rotation policy. The default rotates when a log file reaches 1 GB.

sys_log_roll_num

Number of rotated log files to retain.

sys_log_verbose_level

Verbosity level for VLOG output in the code.

sys_log_verbose_modules

Module filter for log output. For example, setting this to olap prints only logs from the olap module.

log_buffer_level

Log flushing strategy. The default keeps logs in memory.

aws_log_level

Log level for the AWS SDK. Valid values:

enable_metric_calculator

Whether to run the metric calculator to collect BE performance indicators.

enable_system_metrics

Whether to collect system metrics.

periodic_counter_update_period_ms

Update interval for rate and sampling calculators.

status_report_interval

Interval at which profile reports are sent.

report_disk_state_interval_seconds

Interval at which the BE agent reports disk status to the frontend node.

report_tablet_interval_seconds

Interval at which the BE agent reports OLAP tablet information to the frontend node.

report_task_interval_seconds

Interval at which the BE agent reports task signatures to the frontend node.

disk_stat_monitor_interval

Interval between consecutive disk status checks.

max_percentage_of_error_disk

Maximum percentage of damaged disks the storage engine tolerates. When the percentage of failed disks exceeds this value, the BE stops.

scan_context_gc_interval_min

Scheduling interval for the context garbage collection thread.

es_http_timeout_ms

Timeout for HTTP connections to Elasticsearch.

es_scroll_keepalive

Duration for which an Elasticsearch scroll context is kept alive.

external_table_connect_timeout_sec

Connection timeout for external table connections.

custom_config_dir

Directory for be_custom.conf. If the conf/ directory might be overwritten during upgrades, point this to a separate location to protect persisted dynamic configuration.

plugin_path

Directory for BE plugins.

pprof_profile_dir

Directory where pprof profiles are saved.

small_file_dir

Directory where SmallFileMgr stores downloaded files.

query_scratch_dirs

Directories where the BE writes temporary spill data during query execution. Separate multiple directories with semicolons (;).

pull_load_task_dir

Directory for pull load task files.

user_function_dir

Directory for user-defined function (UDF) files.

cluster_id

ID of the cluster this BE belongs to. This value is normally delivered by the frontend node via heartbeat and does not need to be set manually. If you set it manually, make sure the cluster_id file in the data directory matches.

enable_stream_load_record

Whether to record Stream Load history.

enable_token_check

Used for forward compatibility. This item will be removed in a future version.

doris_cgroups

cgroups assigned to the Doris process.

max_client_cache_size_per_host

Maximum number of client connections cached per remote host.

file_descriptor_cache_capacity

Number of file handles the BE caches.

cache_clean_interval

Interval at which unused file handle caches and Segment Cache entries are cleaned.

min_file_descriptor_number

Minimum number of file descriptors the BE requires.

min_cumulative_compaction_num_singleton_deltas

Minimum number of incremental files required to trigger cumulative compaction.

max_cumulative_compaction_num_singleton_deltas

Maximum number of incremental files processed in a single cumulative compaction run.

max_memory_sink_batch_count

Maximum number of batches for external scan caches. Works with batch_size (default 1024 rows): the default settings cache up to 20 × 1024 = 20,480 rows of external scan data.