Host ElasticJob jobs on SchedulerX 2.0

Updated at:
Copy as MD

SchedulerX 2.0 is compatible with the open-source ElasticJob API. You can host your ElasticJob jobs on the SchedulerX 2.0 platform without changing your code. This topic describes how to host ElasticJob jobs on the SchedulerX 2.0 platform.

Background information

ElasticJob is a lightweight, decentralized, distributed job scheduling framework built on Quartz and uses ZooKeeper as its registry center. It is now an open-source project under the Apache Software Foundation. For more information, see ElasticJob.

Advantages of SchedulerX 2.0

Rich visualization capabilities

  • View user dashboards. The Overview page in the SchedulerX console provides visual statistics. The Professional Edition Statistics area displays the total number of jobs, application jobs, disabled jobs, online workers, and currently running job executions. The Professional Edition Job Execution Summary area uses a line chart to show the number of triggers, successful executions, and failed executions over recent hours. The right-side panel includes Resource Overview, Product News, and Quick Links for submitting tickets, accessing product documentation, viewing pricing details, and getting started quickly.

  • View job history. The Job Executions page allows you to filter jobs by Status and groupId. The table displays information such as job ID/name, job type/execution mode, execution ID, groupId, start/end time, and operator. The job status is indicated by a green Success or red Failed tag. For each entry, you can view Details and Logs, or select Rerun.

  • Query execution logs. On the Log Query page, you can filter by Job ID or Search Fields and set a time range, such as Last 15 minutes. Click Query to view the job execution logs, which include fields such as ip, executionId, level, and log.

  • View thread stacks. In the Job Executions list, you can filter job executions by status. Select a running job execution to view its thread stack information in the View Stack panel on the right.

  • View operation records. The operation records page in SchedulerX displays audit information in a table. Columns include ID, Operation Time, Operation Type, Operator, Details, and Actions. You can filter by type and browse through pages. Operation types include deleting or modifying an application group, granting permissions, upgrading an application group, enabling a job, specifying machines, and modifying a job.

Advanced features

  • Job orchestration

    Supports workflow (DAG) orchestration for jobs through a drag-and-drop interface. A detailed job status graph helps you understand the reasons for downstream job failures.6

  • Concurrency throttling

    A common use case is running offline reporting jobs at night. For example, many reporting jobs start at 1:00 AM or 2:00 AM. You must control the maximum number of concurrent jobs for an application to avoid overwhelming your business systems. Jobs that exceed the concurrency limit wait in a queue. If you also require KPI reports to be completed before 9:00 AM, you can set a high priority for KPI jobs. High-priority jobs will preempt low-priority ones, ensuring they are scheduled first.

    SchedulerX 2.0 supports preemptive priority queues for jobs, which can be easily configured in the console.

    7

  • Resource isolation

    Supports namespace-level and application-level resource isolation, along with multi-tenant permission management.

Enterprise-grade alerting and operations

  • Alerting

    Supports notifications for job execution failures, timeouts, and unavailable machines through email, DingTalk, text messages, or phone calls.

  • Operations

    Supports operations such as running jobs in-place, rerun, update job outputs, mark as completed, viewing stacks, stopping jobs, and specifying machines.

Fully managed and cost-effective

ElasticJob depends on ZooKeeper for job storage and scheduling coordination, requiring at least a three-node ZooKeeper cluster. If a node fails, you must reconfigure the ZooKeeper server and client settings, which may require restarting all your applications. Furthermore, due to ZooKeeper's functional limitations, a single ZooKeeper cluster cannot support a large volume of jobs and cannot be horizontally scaled to support a higher TPS. This means you would need to maintain multiple ZooKeeper clusters, increasing infrastructure costs.

By hosting ElasticJob jobs on SchedulerX 2.0, you eliminate the need to maintain your own ZooKeeper clusters and no longer need to worry about the growth in job volume. This saves both infrastructure and operational costs.

High availability

SchedulerX 2.0 uses a high availability architecture with a multi-replica mechanism for jobs. It has been battle-tested over many years in Alibaba Group's Double 11 events and disaster recovery drills. The system can withstand the failure of any two nodes in a cluster or a power outage in an entire data center without affecting job scheduling.

Comparison with open-source ElasticJob

Feature

Open-source ElasticJob

ElasticJob on SchedulerX

Simple jobs

对

对

Script jobs

对

对

Dataflow jobs

对

对

Standalone

对

对

Sharding broadcast

对

对

Time-based scheduling

cron

cron, fixed_rate, fixed_delay, one_time

Workflow

错

对

Visualization

None

Execution history, logging service, thread stacks, operation records, user dashboards, and more.

Monitoring and alerting

Email

Email, DingTalk groups, text messages, phone calls

Operations

None

Run once, rerun, update job outputs, mark as completed, stop jobs

Integrate with Spring Boot

This section uses ElasticJob 3.0.2 as an example. For a complete demo project, see Demo project.

Step 1: Create a namespace and an application

Before you begin, create a namespace and an application in the SchedulerX console. For more information, see Create a namespace and Create an application.

In the SchedulerX console, click Application Management in the left-side navigation pane. In the application list, confirm that your application has been created. You can view details such as the application name, groupId/appKey, version, used/max jobs, and total instances.

Step 2: Add the ElasticJob dependency

In the pom.xml file, add the schedulerx2-plugin-elasticjob dependency above the existing elasticjob dependency.

 <dependency>
     <groupId>com.aliyun.schedulerx</groupId>
     <artifactId>schedulerx2-plugin-elasticjob</artifactId>
     <version>3.0.2.2</version>
</dependency>
<dependency>
    <groupId>org.apache.shardingsphere.elasticjob</groupId>
    <artifactId>elasticjob-lite-spring-boot-starter</artifactId>
    <version>3.0.2</version>
</dependency>

Step 3: Configure the SchedulerX scheduler

In the application.yml file, add the elasticjob.scheduler=schedulerx configuration item and the spring.schedulerx2 configuration.

Note

For a detailed list of SchedulerX configuration parameters, see Connect a Spring Boot application to SchedulerX.

elasticjob:
  regCenter:
    serverLists: 127.0.0.1:2181
    namespace: elasticjob-test
  scheduler: schedulerx # Set the scheduler to schedulerx. Otherwise, the native ElasticJob scheduler is used.
  jobs:
    simpleJob:
      elasticJobClass: com.alibaba.elasticjob.test.processor.SpringBootSimpleJob
      cron: 0/10 * * * * ?
      shardingTotalCount: 1
      overwrite: true
    shardingJob:
      elasticJobClass: com.alibaba.elasticjob.test.processor.SpringBootShardingJob
      cron: 0 * * * * ?
      shardingTotalCount: 3
      shardingItemParameters: 0=Beijing,1=Shanghai,2=Guangzhou
      overwrite: true
# Add the following SchedulerX configuration.
spring:
   schedulerx2:
      endpoint: acm.aliyun.com
      namespace: 433d8b23-xxxx-xxxx-xxxx-90d4d1b9a4af
      groupId: xueren_primary
      appKey: xxxxxxxxxxx
      regionId: public
      aliyunAccessKey: xxxxxxxxxxxx
      aliyunSecretKey: xxxxxxxxxxxx

If you need to switch back to the ElasticJob scheduler, you can use JVM -D options to change it at runtime:

-Dspring.schedulerx2.enabled=false -Delasticjob.scheduler=elasticjob

Step 4: Add a component scan to your startup class

The program now automatically scans com.alibaba.schedulerx.plugin.* at startup. The following is a code example:

@SpringBootApplication
@ComponentScan(value = "com.alibaba.elasticjob.*")
@ComponentScan(value = "com.alibaba.schedulerx.test.*")
@ComponentScan(value = "com.alibaba.schedulerx.plugin.*")
public class SpringBootMain {
    // CHECKSTYLE:OFF
    public static void main(final String[] args) throws InterruptedException {
    // CHECKSTYLE:ON
        SpringApplication.run(SpringBootMain.class, args);
    }
}

Step 5 (Optional): Configure the logging service

You can configure a logging service to collect logs from your client. The following example uses log4j2. For more information, see Logging services for SchedulerX.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="off">
	<Appenders>
		<Console name="Console" target="SYSTEM_OUT">
			<PatternLayout
				pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n" />
		</Console>
		<SchedulerxLog4j2Appender name="schedulerxLog"
            timeFormat="yyyy-MM-dd'T'HH:mmZ"
            timeZone="UTC"
            ignoreExceptions="true">
        <PatternLayout pattern="%d %-5level [%thread] %logger{0}: %msg"/>
    </SchedulerxLog4j2Appender>
	</Appenders>
	<Loggers>
		<Root level="info">
			<AppenderRef ref="Console" />
		</Root>
		<Logger name="schedulerx" level="info" additivity="false">
			<AppenderRef ref="schedulerxLog" />
		</Logger>
	</Loggers>
</Configuration>

The following code provides an example of a Java job class:

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.shardingsphere.elasticjob.api.ShardingContext;
import org.apache.shardingsphere.elasticjob.simple.job.SimpleJob;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import com.alibaba.schedulerx.test.service.HelloService;
@Component
public class SpringBootSimpleJob implements SimpleJob {
    private static final Logger LOGGER = LogManager.getLogger("schedulerx");
    @Autowired
    private HelloService helloService;
    @Override
    public void execute(ShardingContext context) {
        LOGGER.info("jobName:" + context.getJobName() + ", hello:" + helloService.hello());
    }
}

Step 6: Verify the job execution

After starting the application, SchedulerX automatically syncs the jobs to the console. During this process:

  • If a cron expression uses a second-level frequency, the time type is converted to second_delay.

  • If the sharding count is greater than 1, the job runs in sharding mode. Otherwise, it runs in standalone mode.

You can view the job execution logs directly from the console.

The logs show that the three shards of the job shardingJob executed successfully. Each log entry has a level of INFO and contains shardingItem=0, shardingItem=1, shardingItem=2, and shardingTotal=3, which indicates that the sharding job is running correctly on different instances.