Best practices for Alibaba Cloud SchedulerX, covering canary releases, log integration, scaling, throttling, and more.
Best practices
Perform a canary release for scheduled jobs
SchedulerX scheduled jobs support a label-based routing policy. You can configure labels in the SDK to route jobs to workers with a specific label, enabling canary releases. For more information, see Perform a canary release for scheduled jobs.
Connect to Simple Log Service
Simple Log Service records and analyzes key information during system operations, helping you troubleshoot faults and identify potential issues. SchedulerX 2.0 lets you view business logs of each scheduled job, including distributed tasks, in the console by adding a Log4j or Logback configuration. No code modification is required. For more information, see Connect to Simple Log Service.
Enable a single application to support more than 100,000 scheduled jobs
By default, a SchedulerX application supports a maximum of 1,000 jobs. When each scheduled job has a different scheduled time and cannot be handled by MapReduce distributed tasks, this limit may be insufficient. To support more than 100,000 jobs, contact the SchedulerX helpdesk to enable automatic application scale-out, or use a client of version 1.2.1 or later with the shared ContainerPool feature. For more information, see Enable a single application to support more than 100,000 scheduled jobs.
Use priority queues for application-level throttling
During sudden peak scheduling, many day-level jobs may start at the same time, risking a system crash if backend services cannot handle the concurrent load. SchedulerX provides two application-level throttling policies: enable throttling with a queue size limit, or configure a preemptible priority queue. For more information, see Use priority queues for application-level throttling.
Create second-level scheduled jobs
Second-level jobs suit services that require high real-time performance, such as near-real-time continuous polling. The memory grid and second-level scheduling enable continuous processing of large data volumes. Second-level jobs in SchedulerX are a type of timed scheduling that supports simple Java jobs, distributed Java jobs, and script jobs with various execution methods. For more information, see Create second-level scheduled jobs.
Set the data timestamp
A job that runs at 00:30 every day may need to process data from the previous day, requiring the data timestamp to differ from the job's running time. Use the data timestamp to process data that does not correspond to the task execution time. For more information, see Set the data timestamp.
Reload Data
When business requirements change, you may need to rerun jobs for a past period. For example, after adding a database field or discovering errors in last month's data, you can rerun scheduled jobs to reprocess the data. For more information, see Rerun jobs to reprocess data.
Connect to Tracing Analysis
In distributed microservice scenarios, end-to-end tracing provides a visual way to analyze and locate execution details at each service point. Scheduled jobs often have longer execution paths than typical real-time services, making tracing integration valuable for querying task execution times and diagnosing issues. For more information, see Connect to Tracing Analysis.
Specify labels
Labels are used in scenarios such as grayscale releases and stress testing. Within the same application group, you can configure labels in the SDK to route jobs to specific machines. For more information, see Specify labels.
Gracefully shut down a job scheduling application
Scheduled jobs run continuously at a fixed frequency. When an application is restarted for a release, directly shutting it down interrupts ongoing jobs, which can lead to incomplete data or other problems. SchedulerX provides a graceful shutdown feature that waits for all running jobs to complete before taking the process offline. For more information, see Gracefully shut down a job scheduling application.