This topic describes the memory usage categories and memory configuration parameters related to a backend (BE) in StarRocks. This topic also describes how to view the memory usage of a BE.
Memory usage categories
To add or modify memory configuration parameters related to a BE in a StarRocks cluster, perform the following steps: On the EMR on ECS page in the E-MapReduce (EMR) console, find the StarRocks cluster that you want to manage and click Services in the Actions column. On the Services tab, click Configure in the StarRocks section. On the Configure tab of the StarRocks service page, click the be.conf tab. On the be.conf tab, modify the required parameters, or click Add Configuration Item to add the required parameters. For more information, see Manage configuration items.
Label | Metric name | Description | Memory configuration parameter related to a BE |
process | starrocks_be_process_mem_bytes | The memory that is actually used by the BE process. Reserved free memory is excluded. | mem_limit |
query_pool | starrocks_be_query_mem_bytes | The total memory that is used by the query layer of the BE. | N/A |
load | starrocks_be_load_mem_bytes | The total memory that is used by an import task. | load_process_max_memory_limit_bytes and load_process_max_memory_limit_percent |
table_meta | starrocks_be_tablet_meta_mem_bytes | The total memory that is used by the metadata. | N/A |
compaction | starrocks_be_compaction_mem_bytes | The total memory that is used by multi-version memory compaction. | compaction_max_memory_limit and compaction_max_memory_limit_percent |
column_pool | starrocks_be_column_pool_mem_bytes | The column pool memory that is used to accelerate data reads at the storage layer of the BE. | N/A |
page_cache | starrocks_be_storage_page_cache_mem_bytes | The page cache at the storage layer of the BE. | disable_storage_page_cache and storage_page_cache_limit |
chunk_allocator | starrocks_be_chunk_allocator_mem_bytes | The cache per CPU core that is used to accelerate the process of applying for small memory chunks. | chunk_reserved_bytes_limit |
consistency | starrocks_be_consistency_mem_bytes | The memory that is used by a periodic consistency check task. | consistency_max_memory_limit_percent and consistency_max_memory_limit |
schema_change | starrocks_be_schema_change_mem_bytes | The total memory that is used by a schema change task. | memory_limitation_per_thread_for_schema_change |
clone | starrocks_be_clone_mem_bytes | The total memory that is used by a tablet clone task. | N/A |
update | starrocks_be_update_mem_bytes | The total memory that is used by the primary key model. | N/A |
Memory configuration parameters
Memory configuration parameters related to a BE
Parameter | Default value | Description |
mem_limit | 90% | The maximum percentage of total memory that the BE can use on the server on which the BE resides. The hard limit is 90%, and the soft limit is 80%. If the BE is deployed as a standalone node, the parameter is not required. If the BE is deployed with other services that consume more memory, the parameter must be appropriately set. |
load_process_max_memory_limit_bytes | 107374182400 | The load_process_max_memory_limit_bytes parameter specifies the maximum amount of memory that an import task can use. The load_process_max_memory_limit_percent parameter specifies the maximum percentage of memory that an import task can use. The system selects the smaller value between If the memory used by an import task reaches the upper limit, disk flush and backpressure are triggered. |
load_process_max_memory_limit_percent | 30 | |
compaction_max_memory_limit | -1 | The compaction_max_memory_limit parameter specifies the maximum amount of memory that a compaction task can use. The compaction_max_memory_limit_percent parameter specifies the maximum percentage of memory that a compaction task can use. The system selects the smaller value between If the memory used by a compaction task reaches the upper limit, the compaction task fails. |
compaction_max_memory_limit_percent | 100 | |
disable_storage_page_cache | true | Specifies whether to disable the page cache at the storage layer of the BE. This parameter is used together with the storage_page_cache_limit parameter. You can enable the page cache in scenarios in which memory resources are sufficient and a large amount of data is queried. This can accelerate query performance. |
storage_page_cache_limit | 0 | The maximum memory that is available for the page cache at the storage layer of the BE. |
chunk_reserved_bytes_limit | 2147483648 | The cache that is used to accelerate the allocation of small memory chunks. The default limit is 2147483648 bytes (2 GB). You can specify the parameter if memory resources are sufficient. |
consistency_max_memory_limit | 10G | The consistency_max_memory_limit parameter specifies the maximum amount of memory that a consistency check task can use. The consistency_max_memory_limit_percent parameter specifies the maximum percentage of memory that a consistency check task can use. The system selects the smaller value between If the memory used by a consistency check task exceeds the upper limit, the consistency check task fails. |
consistency_max_memory_limit_percent | 20 | |
memory_limitation_per_thread_for_schema_change | 2 | The upper limit of memory that a schema change task can use. If the memory used by a schema change task exceeds the upper limit, the schema change task fails. |
tc_use_memory_min | 10737418240 | The minimum reserved memory for TCMalloc. If the reserved memory for TCMalloc is less than the threshold, StarRocks does not return free memory to the operating system. |
Session parameters
Parameter | Default value | Description |
exec_mem_limit | 2147483648 | The memory limit for a single instance. |
load_mem_limit | 0 | The memory limit for a single import task. The default value is 0, which indicates that the exec_mem_limit parameter is used to specify the memory limit. |
View memory usage
You can use one of the following methods to analyze the memory usage of a BE:
Method 1: Analyze the memory usage of a BE by using the metrics interface
The statistics collected by the metrics interface are updated every 10 seconds. The following example shows how to access the metrics interface by using cURL:
# The default webserver port of a BE is 8040.
curl -XGET -s http://BE_IP:8040/metrics | grep "^starrocks_be_.*_mem_bytes\|^starrocks_be_tcmalloc_bytes_in_use"For more information about the metrics, see Memory usage categories.
Method 2: Analyze the memory usage of a BE by using the mem_tracker interface
You can access the mem_tracker interface by using a browser or cURL. In this example, a browser is used.
# The default webserver port of a BE is 8040.
http://be_ip:8040/mem_trackerThe following figure shows the details of memory usage. 
Parameters:
level: MemTracker is a tree structure. Level 1 represents the total memory used by the BE. Level 2 represents different memory usage categories.
Label: This parameter is used to identify the memory usage category. For more information about the labels, see Memory usage categories.
Parent: the label of the parent node.
Limit: the memory usage limit. A value of none indicates that memory usage is not limited.
Current Consumption: the current memory usage.
Peak Consumption: the memory usage during peak hours.