Configure job throttling
In SchedulerX for XXL-JOB, you can configure job-level and application-level throttling to effectively manage system resources. This prevents system overloads and resource overconsumption from too many concurrent jobs.
Job-level throttling
-
The open source XXL-JOB uses a client-side blocking strategy for job throttling. However, this strategy only ensures serial execution on a single executor.
The XXL-JOB blocking strategy offers three options: Serial execution, Discard later, and Cover early.
-
SchedulerX for XXL-JOB supports server-side job-level throttling. Log on to the SchedulerX for XXL-JOB console and go to your XXL-JOB instance. In the left-side navigation pane, click Task Management. Find the job you want to modify and click Edit in the Actions column. In the Advanced Configuration section, use the Task Concurrency parameter to control how many job instances can run concurrently.
If you set Task Concurrency to 1, the job runs idempotently. If a job has not finished when the next one is scheduled, SchedulerX skips the new execution.
Application-level throttling
If an application runs many jobs concurrently, it can overload backend systems. To prevent this, you can configure application-level throttling.
For example, a data platform application processes the previous day's offline data every night. All jobs must finish by 9:30 a.m. However, if all jobs are scheduled to start at 12:30 a.m. at the same time, the sudden spike in load can crash the backend.
SchedulerX for XXL-JOB uses a preemptive task priority queue to solve this problem.
-
Log on to the SchedulerX for XXL-JOB console. On the Application Management page, edit the target application. In the Advanced Configuration section, enable Flow Control and set the Task Execution Queue Size.
For example, if you set the Task Execution Queue Size to 10, a maximum of 10 jobs can run concurrently within the application. SchedulerX places any additional jobs in a waiting queue.
-
When you create jobs, you can assign a different priority to each one. High-priority jobs preempt lower-priority ones. In the waiting queue, high-priority jobs are placed ahead of their lower-priority counterparts.