Alibaba Cloud released SchedulerX 2.0 for commercial use. SchedulerX 2.0 is compatible with the interfaces of XXL-JOB jobs and supports the @JobHandler annotation and the new @XxlJob annotation. You can use SchedulerX 2.0 to schedule XXL-JOB jobs without the need to modify the code of the jobs. This topic describes the benefits of SchedulerX 2.0 and how to manage XXL-JOB jobs in SchedulerX 2.0.
Benefits
Saved server and O&M costs
A minimum configuration of two servers and one database is required by self-managed XXL-JOB jobs. If you use SchedulerX 2.0 to manage XXL-JOB jobs, you can save costs on servers and O&M personnel.
High compatibility
When you upgrade the SDK, variations between XXL-JOB versions can cause compatibility issues. In this case, code refactoring is required to ensure compatibility. SchedulerX 2.0 is highly compatible with the interfaces and annotation methods across various XXL-JOB versions. This eliminates the need to modify the business code during SDK upgrades.
High capacity and precise scheduling
Open source XXL-JOB jobs generally run on one worker node due to the distributed database lock mechanism, which puts a lot of stress on the database. Statistics indicate that the response latency greatly increases when over 10,000 minute-level XXL-JOB jobs are processed by a single worker. When XXL-JOB jobs are processed at a second granularity, the response latency increases even further.
SchedulerX 2.0 employs a distributed architecture that assigns jobs to individual worker nodes. This eliminates contention for locks among processes. This design facilitates horizontal scalability, enabling the platform to schedule millions of concurrent jobs. SchedulerX 2.0 uses a dedicated architecture optimized for minimal resource consumption. This enables low-latency execution of jobs that need to be triggered at a second-level granularity. This feature makes SchedulerX 2.0 suitable for real-time applications that require scheduling at a second-level granularity. SchedulerX 2.0 also supports one-time jobs. This allows users to schedule and execute a job only once at a scheduled point in time. After the job is executed, SchedulerX 2.0 automatically deletes it. This capability applies to scenarios such as sending timed notifications or automatically closing orders at a scheduled deadline.
Powerful data visualization capabilities
View the user dashboard
View the historical records of jobs
View the operational logs
View the running stack for a job
View the historical records of a job
Advanced features
Job orchestration
SchedulerX 2.0 also provides a visually intuitive approach to job orchestration based on Directed Acyclic Graphs (DAGs). You can design complex job dependencies and sequences by simply dragging and dropping nodes on the graphical frontend interface. Within the granular view of a DAG for any given job in SchedulerX 2.0, you can easily access comprehensive job status information and figure out the cause of failed downstream jobs.
Throttling
SchedulerX 2.0 is often used to run overnight jobs for generating offline reports. For example, a large number of report generation jobs start at 01:00 or 02:00. To avoid service interruption, SchedulerX 2.0 limits the number of simultaneous jobs running within an application. If the upper limit is reached, the additional jobs are automatically queued to prevent overload issues. If you want to make sure that KPI reports are completed by 09:00, you can assign a higher priority to the KPI report generation job. This prioritization feature ensures that jobs of higher priorities are completed ahead of jobs of lower priorities.
SchedulerX 2.0 allows jobs in a preemptible queue to be preempted by jobs that have higher priorities. You can enable this feature in the SchedulerX console.
Resource isolation
SchedulerX 2.0 allows you to isolate resources by namespace or application. SchedulerX 2.0 also supports permission management in multitenancy scenarios.
Commercialized alerting and O&M
Alerting
SchedulerX 2.0 sends alerts for job execution issues, including failures, timeouts, and absence of available workers. Multiple alerting methods are supported, such as emails, DingTalk, text messages, and phone calls. You can diagnose the failure causes by examining the details in the alert notifications.
O&M operations
SchedulerX 2.0 allows you to rerun jobs, refresh job data, mark jobs as completed, view stacks, terminate jobs, and specify workers.
Differences between XXL-JOB based on SchedulerX V2.0 and open source XXL-JOB
The following table describes the differences between XXL-JOB based on SchedulerX V2.0 and open source XXL-JOB.
Feature | Open source XXL-JOB | SchedulerX V2.0-based XXL-JOB |
Java jobs | ||
Go jobs | ||
Script jobs | ||
HTTP jobs | ||
Standalone | ||
Sharding broadcast | ||
MapReduce | ||
Routing policies | First, last, round-robin, and random | Round-robin |
Time-based scheduling | cron | cron, fixed_rate, fixed_delay, and one_time |
Workflows | ||
O&M operations | Run one-time jobs and terminate jobs | Run one-time jobs, terminate jobs, rerun jobs, refresh historical data, and mark jobs as completed |
Log query | Logs are stored on on-premises executors. If executors crash down, logs are lost. | Logs of the last two weeks are stored in Logstores of Simple Log Service provided by Alibaba Cloud and support for query. |
Observability | Historical records and dashboard | Historical records, dashboard, operational records, stack view, and tracing analysis |
Monitoring and alerting | Email, DingTalk, Lark, WeCom, custom webhook, text message, and phone call | |
High availability and disaster recovery | Self-responsible for disaster recovery of databases and servers | Support for multi-data center disaster recovery in the same region |
User permissions | User isolation and password-based logon | Support for single sign-on (SSO) and fine-grained permission management of main accounts, Resource Access Management (RAM) users, and RAM roles |
Graceful shutdown | ||
End-to-end canary release | ||
Throttling |
Connect XXL-JOB jobs to SchedulerX 2.0
Procedure
Delete the com.xuxueli:xxl-job-core dependency and add com.aliyun.schedulerx:schedulerx2-plugin-xxljob
, which is the dependency and plug-in of the SchedulerX agent. In this example, schedulerx2-spring-boot-starter is connected.
The XXL-JOB interface is reconstructed in V 2.3.X. Therefore, you must select a connection method based on the XXL-JOB version that you use.
Add the following dependencies to the pom.xml file to connect the SchedulerX agent:
XXL-JOB V2.3.X
<!-- xxl-job-core annotation --> <!-- <dependency> <groupId>com.xuxueli</groupId> <artifactId>xxl-job-core</artifactId> <version>${project.parent.version}</version> </dependency> --> <! -- Add the schedulerx dependency --> <dependency> <groupId>com.aliyun.schedulerx</groupId> <artifactId>schedulerx2-spring-boot-starter</artifactId> <version>${schedulerx2.version}</version> </dependency> <! -- Add schedulerx2-plugin-xxljob --> <dependency> <groupId>com.aliyun.schedulerx</groupId> <artifactId>schedulerx2-plugin-xxljob</artifactId> <version>2.3.5</version> </dependency>
For more information, see Demo.
XXL-JOB V2.2.x or earlier
<!-- xxl-job-core annotation --> <!-- <dependency> <groupId>com.xuxueli</groupId> <artifactId>xxl-job-core</artifactId> <version>${project.parent.version}</version> </dependency> --> <! -- Add the schedulerx dependency --> <dependency> <groupId>com.aliyun.schedulerx</groupId> <artifactId>schedulerx2-spring-boot-starter</artifactId> <version>${schedulerx2.version}</version> </dependency> <! -- Add schedulerx2-plugin-xxljob --> <dependency> <groupId>com.aliyun.schedulerx</groupId> <artifactId>schedulerx2-plugin-xxljob</artifactId> <version>2.2.4</version> </dependency>
For more information, see Demo.
Add the following configurations to the application.properties file:
spring.schedulerx2.endpoint=xxxxxxx spring.schedulerx2.namespace=xxxxxxx spring.schedulerx2.groupId=xxxxxxxx spring.schedulerx2.appKey=xxxxxxx
Create a method job
@XxlJob method annotation
In this section, the XXL-JOB-2.2.1 interface is used as an example and the open source project xxl-job-executor-sample-springboot is referenced to create a method job. The following figure shows the code of the method job.
Create the job in the SchedulerX console.
Run the job once to print the job parameters.
@JobHandler class annotation
Compile the following code.
Create the job in the SchedulerX console.
The following figure shows that the job parameters are printed.
Sharding broadcast
Compile the following code. In this example, the @XxlJob annotation is used.
Create the job in the SchedulerX console.
Launch two agents to run the job once and separately print the job parameters.
Obtain job parameters
If the version of the plug-in that you use is V2.2.x, run the following code to obtain job parameters:
// If you use a V2.2.x plug-in, you can use one of the following methods to obtain job parameters: // 1. Use the String input parameter of the method. // 2. Use the XxlJobHelper.getJobParam() method. @XxlJob("demoJobHandler") public void demoJobHandler(String jobParameters) throws Exception { XxlJobHelper.log("XXL-JOB, Hello World." + jobParameters); String jobParam = XxlJobHelper.getJobParam(); XxlJobHelper.log("XXL-JOB, Hello World." + jobParam); // default success }
If the version of the plug-in that you use is V2.3.x, run the following code to obtain job parameters:
// If you use a V2.3.x plug-in, you can use one of the following methods to obtain job parameters: // 1. Use the XxlJobHelper.getJobParam() method. @XxlJob("demoJobHandler") public void demoJobHandler() throws Exception { String jobParam = XxlJobHelper.getJobParam(); XxlJobHelper.log("XXL-JOB, Hello World." + jobParam); // default success }
Migrate multiple XXL-JOB jobs at a time
Step 1: Export the configurations of XXL-JOB jobs
Download xxljob-export.tar.
Decompress the package, view the readme file, and add the following code to the application.properties file:
### xxl-job, datasource datasource.url=jdbc:mysql://127.0.0.1:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai datasource.username=root datasource.password=123456 datasource.driver-class-name=com.mysql.cj.jdbc.Driver ### Specify the name of the application whose job configurations you want to export. If you skip this step, the configurations of the jobs in all groups are exported. #xxl-job.app-name=xxl-job-executor-sample # Optional. Filter the specified job description, which contains the specified string. #xxl-job.filter.job-desc=test # Optional. Filter the author information of the specified job, which contains the specified string. #xxl-job.filter.author=test # Optional. Filter the configurations of jobs in the specified state (0: disabled; 1: enabled). If you skip this step, the configurations of all jobs are exported. #xxl-job.filter.trigger-status=0 # Optional. Specify the start page number of job configurations to read. Default value: 0. Job configurations are sorted in different pages by job ID. #xxl-job.filter.page-start=0 # Optional. Specify the page size of job configurations to read. By default, the page size has no limit. Job configurations are sorted by job ID. #xxl-job.filter.page-size=10 # Optional. Specify the status of exported jobs. Valid values: 0 (disabled) and 1 (enabled). By default, the original status of jobs before the export is retained. #xxl-job.default.job-status=0
Run the
./start.sh
command to export the job configurations.After the configurations are exported, obtain the xxljob_*******.json configuration file that contains only job configurations in Bean mode.
Step 2: Import XXL-JOB jobs to SchedulerX 2.0
Log on to the SchedulerX console. In the left-side navigation pane, click Task Management. On the Task Management page, click .
Import the exported configuration file to synchronize the XXL-JOB jobs to SchedulerX.
FAQ
jobprocessor is null
If the error occurs, you must initialize XxlJobSpringExecutor. Sample code:
/**
* Copyright (c) 2022-present Alibaba Group
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
package com.alibaba.schedulerx.example.xxljob;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
@Configuration
public class XxlJobConfig {
private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);
@Bean
public XxlJobSpringExecutor xxlJobExecutor() {
logger.info(">>>>>>>>>>> xxl-job config init.");
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
return xxlJobSpringExecutor;
}
}