All Products
Search
Document Center

Microservices Engine:Manage XXL-JOB jobs in SchedulerX 2.0

Last Updated:Aug 26, 2024

Alibaba Cloud released SchedulerX 2.0 for commercial use. SchedulerX 2.0 is compatible with the interfaces of XXL-JOB jobs and supports the @JobHandler annotation and the new @XxlJob annotation. You can use SchedulerX 2.0 to schedule XXL-JOB jobs without the need to modify the code of the jobs. This topic describes the benefits of SchedulerX 2.0 and how to manage XXL-JOB jobs in SchedulerX 2.0.

Benefits

Saved server and O&M costs

A minimum configuration of two servers and one database is required by self-managed XXL-JOB jobs. If you use SchedulerX 2.0 to manage XXL-JOB jobs, you can save costs on servers and O&M personnel.

High compatibility

When you upgrade the SDK, variations between XXL-JOB versions can cause compatibility issues. In this case, code refactoring is required to ensure compatibility. SchedulerX 2.0 is highly compatible with the interfaces and annotation methods across various XXL-JOB versions. This eliminates the need to modify the business code during SDK upgrades.

High capacity and precise scheduling

Open source XXL-JOB jobs generally run on one worker node due to the distributed database lock mechanism, which puts a lot of stress on the database. Statistics indicate that the response latency greatly increases when over 10,000 minute-level XXL-JOB jobs are processed by a single worker. When XXL-JOB jobs are processed at a second granularity, the response latency increases even further.

SchedulerX 2.0 employs a distributed architecture that assigns jobs to individual worker nodes. This eliminates contention for locks among processes. This design facilitates horizontal scalability, enabling the platform to schedule millions of concurrent jobs. SchedulerX 2.0 uses a dedicated architecture optimized for minimal resource consumption. This enables low-latency execution of jobs that need to be triggered at a second-level granularity. This feature makes SchedulerX 2.0 suitable for real-time applications that require scheduling at a second-level granularity. SchedulerX 2.0 also supports one-time jobs. This allows users to schedule and execute a job only once at a scheduled point in time. After the job is executed, SchedulerX 2.0 automatically deletes it. This capability applies to scenarios such as sending timed notifications or automatically closing orders at a scheduled deadline.

Powerful data visualization capabilities

View the user dashboard

View the historical records of jobs

View the operational logs

View the running stack for a job

View the historical records of a job

Advanced features

Job orchestration

SchedulerX 2.0 also provides a visually intuitive approach to job orchestration based on Directed Acyclic Graphs (DAGs). You can design complex job dependencies and sequences by simply dragging and dropping nodes on the graphical frontend interface. Within the granular view of a DAG for any given job in SchedulerX 2.0, you can easily access comprehensive job status information and figure out the cause of failed downstream jobs.

6

Throttling

SchedulerX 2.0 is often used to run overnight jobs for generating offline reports. For example, a large number of report generation jobs start at 01:00 or 02:00. To avoid service interruption, SchedulerX 2.0 limits the number of simultaneous jobs running within an application. If the upper limit is reached, the additional jobs are automatically queued to prevent overload issues. If you want to make sure that KPI reports are completed by 09:00, you can assign a higher priority to the KPI report generation job. This prioritization feature ensures that jobs of higher priorities are completed ahead of jobs of lower priorities.

SchedulerX 2.0 allows jobs in a preemptible queue to be preempted by jobs that have higher priorities. You can enable this feature in the SchedulerX console.

7

Resource isolation

SchedulerX 2.0 allows you to isolate resources by namespace or application. SchedulerX 2.0 also supports permission management in multitenancy scenarios.

Commercialized alerting and O&M

Alerting

SchedulerX 2.0 sends alerts for job execution issues, including failures, timeouts, and absence of available workers. Multiple alerting methods are supported, such as emails, DingTalk, text messages, and phone calls. You can diagnose the failure causes by examining the details in the alert notifications.

O&M operations

SchedulerX 2.0 allows you to rerun jobs, refresh job data, mark jobs as completed, view stacks, terminate jobs, and specify workers.

2

Differences between XXL-JOB based on SchedulerX V2.0 and open source XXL-JOB

The following table describes the differences between XXL-JOB based on SchedulerX V2.0 and open source XXL-JOB.

Feature

Open source XXL-JOB

SchedulerX V2.0-based XXL-JOB

Java jobs

对

对

Go jobs

对

对

Script jobs

对

对

HTTP jobs

错

对

Standalone

对

对

Sharding broadcast

对

对

MapReduce

错

对

Routing policies

First, last, round-robin, and random

Round-robin

Time-based scheduling

cron

cron, fixed_rate, fixed_delay, and one_time

Workflows

错

对

O&M operations

Run one-time jobs and terminate jobs

Run one-time jobs, terminate jobs, rerun jobs, refresh historical data, and mark jobs as completed

Log query

Logs are stored on on-premises executors. If executors crash down, logs are lost.

Logs of the last two weeks are stored in Logstores of Simple Log Service provided by Alibaba Cloud and support for query.

Observability

Historical records and dashboard

Historical records, dashboard, operational records, stack view, and tracing analysis

Monitoring and alerting

Email

Email, DingTalk, Lark, WeCom, custom webhook, text message, and phone call

High availability and disaster recovery

Self-responsible for disaster recovery of databases and servers

Support for multi-data center disaster recovery in the same region

User permissions

User isolation and password-based logon

Support for single sign-on (SSO) and fine-grained permission management of main accounts, Resource Access Management (RAM) users, and RAM roles

Graceful shutdown

对

对

End-to-end canary release

错

对

Throttling

错

对

Connect XXL-JOB jobs to SchedulerX 2.0

Procedure

Delete the com.xuxueli:xxl-job-core dependency and add com.aliyun.schedulerx:schedulerx2-plugin-xxljob, which is the dependency and plug-in of the SchedulerX agent. In this example, schedulerx2-spring-boot-starter is connected.

Note

The XXL-JOB interface is reconstructed in V 2.3.X. Therefore, you must select a connection method based on the XXL-JOB version that you use.

  1. Add the following dependencies to the pom.xml file to connect the SchedulerX agent:

    XXL-JOB V2.3.X

    <!-- xxl-job-core annotation -->
    <!--
    <dependency>
    	<groupId>com.xuxueli</groupId>
    	<artifactId>xxl-job-core</artifactId>
    	<version>${project.parent.version}</version>
    </dependency>
    -->
    <! -- Add the schedulerx dependency -->
    <dependency>
      <groupId>com.aliyun.schedulerx</groupId>
      <artifactId>schedulerx2-spring-boot-starter</artifactId>
      <version>${schedulerx2.version}</version>
    </dependency>
    <! -- Add schedulerx2-plugin-xxljob -->
    <dependency>
    	<groupId>com.aliyun.schedulerx</groupId>
    	<artifactId>schedulerx2-plugin-xxljob</artifactId>
    	<version>2.3.5</version>
    </dependency>

    For more information, see Demo.

    XXL-JOB V2.2.x or earlier

    <!-- xxl-job-core annotation -->
    <!--
    <dependency>
    	<groupId>com.xuxueli</groupId>
    	<artifactId>xxl-job-core</artifactId>
    	<version>${project.parent.version}</version>
    </dependency>
    -->
    <! -- Add the schedulerx dependency -->
    <dependency>
      <groupId>com.aliyun.schedulerx</groupId>
      <artifactId>schedulerx2-spring-boot-starter</artifactId>
      <version>${schedulerx2.version}</version>
    </dependency>
    <! -- Add schedulerx2-plugin-xxljob -->
    <dependency>
    	<groupId>com.aliyun.schedulerx</groupId>
    	<artifactId>schedulerx2-plugin-xxljob</artifactId>
    	<version>2.2.4</version>
    </dependency>

    For more information, see Demo.

  2. Add the following configurations to the application.properties file:

    spring.schedulerx2.endpoint=xxxxxxx
    spring.schedulerx2.namespace=xxxxxxx
    spring.schedulerx2.groupId=xxxxxxxx
    spring.schedulerx2.appKey=xxxxxxx

Create a method job

@XxlJob method annotation

  1. In this section, the XXL-JOB-2.2.1 interface is used as an example and the open source project xxl-job-executor-sample-springboot is referenced to create a method job. The following figure shows the code of the method job.1

  2. Create the job in the SchedulerX console.1

  3. Run the job once to print the job parameters.

    2

@JobHandler class annotation

  1. Compile the following code.4

  2. Create the job in the SchedulerX console.5

  3. The following figure shows that the job parameters are printed.6

Sharding broadcast

  1. Compile the following code. In this example, the @XxlJob annotation is used.6

  2. Create the job in the SchedulerX console.7

  3. Launch two agents to run the job once and separately print the job parameters.8

Obtain job parameters

  • If the version of the plug-in that you use is V2.2.x, run the following code to obtain job parameters:

    // If you use a V2.2.x plug-in, you can use one of the following methods to obtain job parameters: 
    // 1. Use the String input parameter of the method. 
    // 2. Use the XxlJobHelper.getJobParam() method. 
    @XxlJob("demoJobHandler")
    public void demoJobHandler(String  jobParameters) throws Exception {
        XxlJobHelper.log("XXL-JOB, Hello World." + jobParameters);
        String jobParam = XxlJobHelper.getJobParam();
        XxlJobHelper.log("XXL-JOB, Hello World." + jobParam);
        // default success
    }
  • If the version of the plug-in that you use is V2.3.x, run the following code to obtain job parameters:

    // If you use a V2.3.x plug-in, you can use one of the following methods to obtain job parameters: 
    // 1. Use the XxlJobHelper.getJobParam() method. 
    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
        String jobParam = XxlJobHelper.getJobParam();
        XxlJobHelper.log("XXL-JOB, Hello World." + jobParam);
        // default success
    }

Migrate multiple XXL-JOB jobs at a time

Step 1: Export the configurations of XXL-JOB jobs

  1. Download xxljob-export.tar.

  2. Decompress the package, view the readme file, and add the following code to the application.properties file: 9

    ### xxl-job, datasource
    datasource.url=jdbc:mysql://127.0.0.1:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
    datasource.username=root
    datasource.password=123456
    datasource.driver-class-name=com.mysql.cj.jdbc.Driver
    
    ### Specify the name of the application whose job configurations you want to export. If you skip this step, the configurations of the jobs in all groups are exported.
    #xxl-job.app-name=xxl-job-executor-sample
    
    # Optional. Filter the specified job description, which contains the specified string.
    #xxl-job.filter.job-desc=test
    
    # Optional. Filter the author information of the specified job, which contains the specified string.
    #xxl-job.filter.author=test
    
    # Optional. Filter the configurations of jobs in the specified state (0: disabled; 1: enabled). If you skip this step, the configurations of all jobs are exported.
    #xxl-job.filter.trigger-status=0
    
    # Optional. Specify the start page number of job configurations to read. Default value: 0. Job configurations are sorted in different pages by job ID.
    #xxl-job.filter.page-start=0
    
    # Optional. Specify the page size of job configurations to read. By default, the page size has no limit. Job configurations are sorted by job ID.
    #xxl-job.filter.page-size=10
    
    
    # Optional. Specify the status of exported jobs. Valid values: 0 (disabled) and 1 (enabled). By default, the original status of jobs before the export is retained.
    #xxl-job.default.job-status=0
  3. Run the ./start.sh command to export the job configurations.11

  4. After the configurations are exported, obtain the xxljob_*******.json configuration file that contains only job configurations in Bean mode.

Step 2: Import XXL-JOB jobs to SchedulerX 2.0

  1. Log on to the SchedulerX console. In the left-side navigation pane, click Task Management. On the Task Management page, click Import Task. 12

  2. Import the exported configuration file to synchronize the XXL-JOB jobs to SchedulerX.

FAQ

jobprocessor is null

If the error occurs, you must initialize XxlJobSpringExecutor. Sample code:

/**
 * Copyright (c) 2022-present Alibaba Group
 *
 * This source code is licensed under the MIT license found in the
 * LICENSE file in the root directory of this source tree.
 */

package com.alibaba.schedulerx.example.xxljob;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
@Configuration
public class XxlJobConfig {
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        logger.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        return xxlJobSpringExecutor;
    }
}