Split flow control limits the number of concurrent split scans running in a task or on a node. Without this limit, high scan concurrency can cause the following issues:
For internal tables, concurrent split scans compete for I/O resources of storage nodes, increasing CPU utilization and memory usage and causing storage nodes to become unstable.
For external tables, scan efficiency varies based on the data source. If the number of concurrent split scans exceeds the quota, new split scans consume the resources of compute nodes and affect other queries instead of speeding up the overall scan.
Split flow control is enabled by default.
This topic describes how to enable or disable split flow control and how to configure the concurrent split scan quota at both the task and node levels.
Prerequisites
Before you begin, ensure that you have:
An AnalyticDB for MySQL cluster running V3.1.10.0 or later
To check the minor version of a cluster, see How do I query the version of an AnalyticDB for MySQL cluster?. To upgrade the minor version, contact technical support.
How it works
When AnalyticDB for MySQL executes a query, it breaks the query into stages. Each stage runs as multiple tasks across nodes. To scan a table, the table is divided into splits. Each split is assigned to a task, and a task can process multiple splits concurrently.
Split flow control operates at two levels:
Task level: Each task has a quota for concurrent split scans. When the number of running split scans in a task reaches the quota, new split scans wait until a running scan completes.
Node level: Each node has an overall quota. The per-task quota adjusts dynamically based on the node quota — increasing when the node has spare capacity and decreasing when the node is under load.
Choose a quota management approach
Before configuring quotas, decide whether to let the system adjust per-task quotas automatically or set them to a fixed value.
| Approach | How it works | When to use |
|---|---|---|
| Dynamic adjustment (recommended) | The system scales each task's quota up or down based on the node's current load | Mixed workloads where resource availability fluctuates |
| Fixed quota | Per-task quota is pinned to the TARGET_RUNNING_SPLITS_LIMIT_PER_TASK value | Predictable workloads where you need precise concurrency control |
Dynamic adjustment requires split flow control to be enabled.
Enable or disable split flow control
Split flow control is enabled by default. To disable or re-enable it, run one of the following statements.
For a cluster:
SET ADB_CONFIG SPLIT_FLOW_CONTROL_ENABLED=true|false;For a single query (using a hint):
/*+ SPLIT_FLOW_CONTROL_ENABLED=true|false*/ SELECT * FROM table;To verify whether split flow control is currently enabled:
SHOW ADB_CONFIG KEY=SPLIT_FLOW_CONTROL_ENABLED;Configure per-task quota
Enable or disable dynamic adjustment
When dynamic adjustment is enabled, each task's quota scales up when the node has spare capacity and scales down when the node quota is under pressure. The per-task quota stays within the bounds set by MIN_RUNNING_SPLITS_LIMIT_PER_TASK and MAX_RUNNING_SPLITS_LIMIT_PER_TASK.
When dynamic adjustment is disabled, the per-task quota is fixed at the TARGET_RUNNING_SPLITS_LIMIT_PER_TASK value (default: 32).
Enable split flow control before enabling dynamic adjustment. Run SHOW ADB_CONFIG KEY=SPLIT_FLOW_CONTROL_ENABLED; to confirm.
To enable or disable dynamic adjustment:
SET ADB_CONFIG NODE_LEVEL_SPLIT_FLOW_CONTROL_ENABLED=true|false;Quota parameters
The following parameters control the per-task concurrent split scan quota.
| Parameter | Default | Valid range | Description |
|---|---|---|---|
MIN_RUNNING_SPLITS_LIMIT_PER_TASK | 1 | 1 to TARGET value | The minimum per-task quota. Under high node load, the dynamic quota decreases to no less than this value. |
TARGET_RUNNING_SPLITS_LIMIT_PER_TASK | 32 | MIN value to MAX value | The baseline per-task quota. When the sum of all tasks' target quotas on a node fits within the node quota, each task's quota increases dynamically. Otherwise, it decreases. This value also sets the fixed quota when dynamic adjustment is disabled. |
MAX_RUNNING_SPLITS_LIMIT_PER_TASK | 64 | Greater than TARGET value | The maximum per-task quota. When the node has spare capacity, the dynamic quota increases to no more than this value. |
Apply quota settings for a cluster or query
Apply quota settings at the cluster level using SET ADB_CONFIG, or at the query level using a hint.
Cluster level:
SET ADB_CONFIG <parameter>=<value>;Query level:
/*+ <parameter>=<value>*/ SELECT * FROM orders;Examples
Set the minimum concurrent split scan quota to 24 for all tasks in the cluster:
SET ADB_CONFIG MIN_RUNNING_SPLITS_LIMIT_PER_TASK=24;Set the minimum quota to 10 for a specific query:
/*+ MIN_RUNNING_SPLITS_LIMIT_PER_TASK=10*/ SELECT * FROM orders;Set the maximum quota to 128 for all tasks in the cluster:
SET ADB_CONFIG MAX_RUNNING_SPLITS_LIMIT_PER_TASK=128;Set the maximum quota to 100 for a specific query:
/*+ MAX_RUNNING_SPLITS_LIMIT_PER_TASK=100*/ SELECT * FROM adb_test;Tuning guidance
Reducing a task's quota means more splits wait in queue, which increases scan latency for that query but frees resources for other queries. Increasing the quota has the opposite effect.
| Scenario | Query characteristics | Recommended approach |
|---|---|---|
| Point queries or short-latency queries | Few splits, small data volume, fast completion expected | Set MIN, TARGET, and MAX to larger values, or disable split flow control for the query |
| Large full-table scans with lower priority | Many splits, long-running, not latency-sensitive | Set TARGET to a smaller value to protect high-priority queries; the quota auto-increases when resources are idle |
| External table queries | Scan efficiency depends on data source throughput | Set the quota based on the data source's transmission limits to avoid overloading the source |
Configure node-level quota
Each node has an overall quota for concurrent split scans. The default is 256 for both storage nodes and compute nodes.
Do not change the default value. An excessively high or low quota degrades cluster performance.
Storage nodes (internal table scans):
SET ADB_CONFIG WORKER_MAX_RUNNING_SOURCE_SPLITS_PER_NODE=256;Compute nodes (external table scans):
SET ADB_CONFIG EXECUTOR_MAX_RUNNING_SOURCE_SPLITS_PER_NODE=256;To query the current node-level quota:
SHOW ADB_CONFIG;For more information, see SHOW ADB_CONFIG.
Usage notes
Node quota changes apply to pending splits only. Decreasing the node quota takes effect only after currently running split scans complete.
Per-task minimum quotas can cause the node quota to be exceeded. If the sum of
MIN_RUNNING_SPLITS_LIMIT_PER_TASKacross all tasks on a node exceeds the node quota, the actual number of concurrent split scans on the node may exceed the configured node quota.