By default, each SchedulerX application supports up to 1,000 jobs. When workloads such as IoT device control or alert rule evaluation require 10,000 to 100,000+ standalone periodic jobs, each with its own schedule, you can enable auto scaling and a shared container pool to remove this limit.
Use cases
Standalone periodic jobs are required when each job must run on its own schedule. MapReduce distributed jobs do not meet this requirement because all tasks within a MapReduce job are triggered at the same time.
The following scenarios typically require 10,000 to 100,000+ standalone periodic jobs:
| Scenario | Description |
|---|---|
| IoT device control | Each IoT switch turns a device on or off at a different time. A standalone periodic job handles the schedule for each switch independently. |
| Alert rule evaluation | The system matches monitoring data against alert rules at an interval of 1 minute. Simple and complex rules vary in execution time. Assigning each rule to a standalone job prevents slow rules from blocking others. |
| Enterprise job scheduling platforms | When you build a scheduling platform on SchedulerX and expose a PoP API for job creation, users across the enterprise may collectively create 10,000 to 100,000+ jobs. |
How it works
When you enable auto scaling for an application, SchedulerX automatically creates a sub-application when the job count reaches 1,000.
At this scale, you must also enable the shared container pool. Without it, the agent creates a separate container pool for every job execution, which quickly exhausts resources. A shared container pool lets all jobs reuse a single pool of execution containers.
Prerequisites
Before you begin, make sure that you have:
A SchedulerX application connected to the SchedulerX service. For setup instructions, see Connect a Spring Boot application to SchedulerX. For other application types, see the topics under
Agent version 1.2.1 or later (earlier versions do not support the shared container pool)
Enable auto scaling and the shared container pool
Step 1: Request auto scaling
Contact SchedulerX technical support to enable auto scaling for your application.
After auto scaling is enabled, SchedulerX automatically creates a sub-application when the job count in the current application reaches 1,000.
Step 2: Update the agent dependency
In the pom.xml file, set the SchedulerX agent version to 1.2.1 or later.
Agent versions earlier than 1.2.1 do not support the shared container pool. Update the agent before you proceed.
Step 3: Configure the shared container pool
Add the following properties to your Spring Boot configuration file:
spring.schedulerx2.shareContainerPool=true # Enable all jobs to share a single container pool
spring.schedulerx2.sharePoolSize=128 # Set the container pool sizeDo not skip this step. Without the shared container pool, the agent creates a new container pool for every job trigger. This can overload the agent and cause failures.
Limits and defaults
| Item | Default | Description |
|---|---|---|
| Jobs per application | 1,000 | When reached, auto scaling creates a new sub-application. Contact technical support to enable auto scaling. |
| Agent version | 1.2.1+ | Minimum version required for the shared container pool. |
sharePoolSize | 128 | Number of containers in the shared pool. |