Running too many jobs simultaneously can overload backend systems, causing resource exhaustion and failed executions. SchedulerX for XXL-JOB provides two levels of throttling to control job concurrency:
Job-level throttling: Limits how many instances of a single job run concurrently. Use this to enforce idempotent execution or cap per-job parallelism.
Application-level throttling: Limits the total number of concurrent jobs across an entire application. Use this when hundreds of jobs trigger at the same time and the backend cannot handle the load.
| Scenario | Throttling level | What to configure |
|---|---|---|
| Prevent duplicate executions of a long-running job | Job-level | Set Task Concurrency to 1 |
| Allow limited parallelism for a single job | Job-level | Set Task Concurrency to the desired limit |
| Prevent backend overload when many jobs trigger simultaneously | Application-level | Enable Flow Control and set Task Execution Queue Size |
| Prioritize high-priority jobs over less urgent ones | Application-level | Enable Flow Control and assign job priorities |
How job-level throttling works
Open-source XXL-JOB provides a blocking strategy that throttles at the client level. This approach runs jobs serially on a single executor, but cannot guarantee global idempotence when multiple executors are registered -- two executors may pick up the same job simultaneously.
SchedulerX for XXL-JOB solves this by supporting job-level throttling that ensures global job idempotence, even when multiple executors are configured.
Runtime behavior: If a job execution lasts until the next scheduling time, the next scheduling is skipped.
Configure job-level throttling
Log on to the SchedulerX for XXL-JOB console.
Find the target instance and click its ID.
In the left-side navigation pane, click Task Management.
Find the target job and click Edit in the Operation column.
In the Advanced Configuration section of the Basic configuration step, set the Task Concurrency parameter.
Task Concurrency controls the maximum number of concurrent executions for a single job:
| Value | Behavior |
|---|---|
| 1 | Enforces idempotent execution. If a job is still running when the next trigger fires, the new execution is skipped. |
| N (greater than 1) | Allows up to N parallel executions of the same job. |
How application-level throttling works
When an application contains many jobs that all trigger at the same time, the backend can become overloaded. For example, a data platform application has hundreds of batch jobs that process the previous day's offline data. All jobs are scheduled to start at 00:30:00 and must finish by 09:30. Triggering them all at once can crash the backend.
Application-level throttling addresses this with a queuing mechanism and preemptive scheduling:
A configurable queue limits how many jobs run concurrently across the application.
Excess jobs wait in a queue rather than overwhelming the backend.
Higher-priority jobs preempt lower-priority jobs, so important data pipelines complete first.
Runtime behavior: When the number of running jobs reaches the Task Execution Queue Size limit, any newly triggered jobs are placed in a waiting queue. As running jobs complete and slots become available, the highest-priority job in the waiting queue is dispatched next.
Configure application-level throttling
Log on to the SchedulerX for XXL-JOB console.
Find the target instance and click its ID.
On the Application Management page, find the target application and click Edit in the Operation column.
In the Advanced Configuration section, turn on Flow Control.
Set the Task Execution Queue Size parameter.
For example, setting Task Execution Queue Size to 10 allows a maximum of 10 jobs to run concurrently. Any additional triggered jobs are placed in a waiting queue.
Assign job priorities
After you enable application-level throttling, assign a priority to each job to control execution order in the queue.
Higher-priority jobs preempt lower-priority jobs. When a slot becomes available, the highest-priority job in the waiting queue is dispatched first.