SchedulerX:Functions and features

Periodic scheduling

• Crontab

  SchedulerX supports periodic scheduling by using UNIX crontab expressions. You cannot use crontab scheduling to trigger a job within seconds. For more information, see Cron.

• Fixed rate

  The interval specified by a crontab expression must be divisible by 60. You cannot use a crontab expression to specify an interval that cannot be divided by 60. For example, if you want to trigger a job at an interval of 40 minutes, you cannot use crontab expressions. Fixed-rate scheduling is suitable for round-robin scheduling and fixed-rate expressions are simple. You cannot use fixed-rate scheduling to trigger a job within seconds. For more information, see Fixed rate.

• Second delay

  Second-delay scheduling is suitable for workloads that require high real-time performance. For example, you can use second-delay scheduling if you want to trigger a job at an interval of 10 seconds. You can use second-delay scheduling to trigger a job within seconds. For more information, see Second delay.

• Calendar

  SchedulerX provides multiple types of calendars for you to schedule jobs. You can also import custom calendars. Calendar scheduling is suitable for financial transactions. For example, you can use calendar scheduling if you want to trigger jobs on each trading day.

• Time zone

  Time zone scheduling is suitable for cross-border business. You can use time zone scheduling if you want to trigger jobs in the time zone of a specific country.

• Data timestamp

  SchedulerX allows you to process service data across days. When you create a job, you can configure a time offset for the job. For example, a job is triggered at 00:30:00 every day. If you want to use the job to process data that was updated at 23:30:00 on the previous day, you must set the time offset for the job to 1 hour. The job is still triggered at 00:30:00 every day. After the job is triggered, the job uses context.getDataTime() to retrieve the data that was updated at 23:30:00 on the previous day.

For more information about how to create a periodic scheduling job, see Create a job.

Job orchestration

You can drag and drop jobs in a directed acyclic graph (DAG) in the SchedulerX console to create a workflow. The DAG provides details of each job and helps you troubleshoot failed downstream jobs. For more information, see Create a workflow.

Diverse job types

The periodic scheduling and workflow scheduling features provided by SchedulerX support different programming languages.

Distributed computing

SchedulerX provides easy-to-use distributed programming models that allow you to perform batch processing on a large amount of data.

• Standalone model

  This model randomly distributes a job to a worker. For more information, see Standalone.

• Broadcast model

  This model runs a job on all workers at the same time and waits until all workers complete the job. For more information, see Broadcast.

• Map model

  This model uses a Map method that is similar to the Map method in Hadoop MapReduce. The Map method allows you to process a large amount of data on multiple workers by using a few lines of code. For more information, see Visual MapReduce.

• MapReduce model

  This model is an extension of the Map model. The MapReduce model deprecates the postProcess method and provides the Reduce interface. After a worker completes all tasks of a job, you can call the Reduce method to return the execution result of the job instance or call back your business data. For more information, see MapReduce.

• Sharding model

  This model is similar to the Elastic-Job model and supports multiple programming languages. You can configure sharding parameters in the SchedulerX console to evenly distribute shards to multiple agents. For more information, see Sharding models for different programming languages.

Automatic retry on failure

• Automatic retry for failed instances

  You can configure the number of retries and a retry interval for a failed job instance in the Advanced Configuration section of the Task Management module. For example, you can set the number of retries to 3 and the retry interval to 30 seconds. In this case, the instance enters the Failed state and an alert is sent after the job fails three retries.

• Automatic retry for failed tasks

  For distributed jobs, such as jobs in the sharding or MapReduce model, SchedulerX also supports retries for the tasks of these jobs if the tasks fail. You can configure the number of retries and a retry interval for failed tasks of a job in the Advanced Configuration section of the Task Management module.

Multiple throttling methods

• Instance concurrency

  SchedulerX implements traffic throttling at a job granularity. Instance concurrency specifies the maximum number of instances that can run in a job at the same time. The default concurrency value is 1. A value of 1 indicates that a job can run only after the job completes the previous run.

• Task concurrency for a single machine

  To throttle traffic of a distributed job, you can specify the number of concurrent tasks on each machine.

• Task concurrency for all machines

  When a large number of machines run in a distributed job, the traffic of tasks of a specific machine may be relatively higher than that of other machines. To throttle traffic of the distributed job, you can configure the pull model in the SchedulerX console. If you specify the number of concurrent tasks on all machines, all compute nodes pull tasks from the master node.

• Application priority

  SchedulerX implements traffic throttling at an application granularity. Application priority specifies the maximum number of jobs that can run at the same time in an application.

Observability

SchedulerX visualizes data on different GUIs. The following figures show specific GUIs.

• User dashboard

• Execution records of a job

• Operational logs of a job

• Stacks of a job

• Traces of a job. For more information, see Integrate tracing analysis.

• Operation records of a job

Powerful O&M capabilities

• Enable and disable jobs

  You can dynamically enable and disable jobs in the SchedulerX console.

• Rerun failed jobs

  If a job fails, you can modify the code of the job, republish the job, and then immediately rerun the job.

• Mark jobs as completed

  If a job fails, you can rerun it after it is calibrated in the background. However, the rerun process takes hours to complete. You can mark the job as completed to skip it.

• Stop jobs

  You can implement kill() of the JobProcessor to terminate jobs or tasks in the SchedulerX console.

• Update job outputs

  You may want to update the output of a job that has a data timestamp. For example, a business report is generated from a workflow. However, you want to add a field to the report or you have found errors in the data of the previous month. In this case, you can trigger the job to reprocess the data of the previous month. SchedulerX allows you to update the outputs of jobs or workflows. The interval of the data points that you want to update must be one day. Different job instances have different data timestamps.

Monitoring and alerting

• Alert type: generates alerts when operations fail to be performed, operations time out, or no machines are available, and sends notifications for successful operations.

• Alert contact: supports alert contact groups.

• Alert record: records historical alerts.

• Alerting method: supports text messages, phone calls, emails, and webhooks.