Microservices Engine:Task management

Create a scheduling task

1. Log on to the MSE SchedulerX console.

2. In the navigation pane on the left, click Task Management and select a region and Namespace.

3. Click Create Task. In the Basic Configuration step, configure the parameters and click Next.

   The following table describes the basic configuration parameters.

   Configuration name	Description
   Task Name	A custom name for the job.
   Description	The description of the job. Specify a concise description to facilitate job searches.
   Application ID	The group to which the job belongs. Select a value from the drop-down list.
   Task Type	The language in which the job is implemented. Valid values: Java, XXL-JOB, SpringSchedule, Golang, ScriptTask, Http, and DataWorks. When you select Script Task, you must select a scripting language: shell, python, or go. You can write the task script in the editor.
   Class Full Path	Note This parameter is available only if the Task Type parameter is set to Java. The full path of the JobProcessor. Example: xxx.xxx.xxx.HelloProcessor.
   Execution Mode	The mode for job execution. The following modes are supported: Stand-alone Operation: The job is executed on a random worker. Broadcast Run: The job is concurrently executed on all workers, and the system waits until all workers complete the job. Visual MapReduce: A Map model that supports fewer than 300 subtasks and provides a subtask list. In Professional Edition, a maximum of 1,000 jobs are allowed, and jobs can be queried by business keyword. MapReduce: A standard MapReduce model for processing many subtasks in parallel. You can only query summary information for the subtasks. Use this mode for jobs with fewer than 1,000,000 subtasks. Shard Run: Similar to the elastic-job model. Shards are evenly distributed to multiple clients for execution based on the configured sharding parameters. This mode supports multiple programming languages. Note The parameters in Advanced Settings vary depending on the selected Execution Mode.
   Task Parameters	An arbitrary string that can be obtained from the context at runtime.

   The following table describes the advanced configuration parameters.

   Parameter	Applicable execution mode	Description	Default value
   Task failure retry count	All modes	The number of automatic retries if a job fails. Note If a job is running on a worker and the worker is restarted, the job fails. If you want to immediately rerun the job, you can specify this parameter.	0
   Task failure retry interval	All modes	The interval between two consecutive retries. Unit: seconds.	30
   Task concurrency	All modes	The number of instances that run the same job at the same time. A value of 1 specifies that concurrent execution is not allowed.	1
   Cleaning strategy	All modes	The cleanup policy for job execution history.	Keep last N entries
   Retained Number	All modes	The number of retained job execution records.	300
   Number of single-machine concurrent subtasks	Visual MapReduce MapReduce Shard run	The number of tasks that concurrently run on a single worker in a distributed model. To speed up the execution, you can specify a larger value. If the downstream or the databases cannot withstand the value that you specified, you can specify a smaller value.	5
   Number of failed retries of subtasks	Visual MapReduce MapReduce Shard run	The number of automatic retries if a task fails in a distributed model.	0
   Sub-task failure retry interval	Visual MapReduce MapReduce Shard run	The interval between two consecutive retries if a task fails in a distributed model. Unit: seconds.	0
   Subtask Failover Strategy	Visual MapReduce MapReduce Shard run	Specifies whether to distribute a task to a new worker after the worker fails to execute the task and is stopped. If you turn on the switch, the system may execute a task more than once when a failover is triggered. You must implement the idempotence of tasks. Note This parameter is applicable to agents V1.8.13 and later.
   The master node participates in the execution	Visual MapReduce MapReduce Shard run	Specifies whether the master node participates in the execution of tasks. At least two workers must be available to run tasks. If an extremely large number of tasks exist, we recommend that you turn off the switch. Note This parameter is applicable to agents V1.8.13 and later.
   Subtask distribution method	Visual MapReduce MapReduce Shard run	Push model: Tasks are evenly distributed to workers. Pull model: Every worker pulls tasks. The Wooden Bucket Theory is not applicable to this model. During the pull process, all tasks are cached on the master node. This puts pressure on the memory. We recommend that you do not distribute more than 10,000 tasks at a time.	Push model
   distribution policy	Visual MapReduce MapReduce	The task distribution policy. Default value: Polling Scheme. Polling Scheme: The system evenly distributes the same number of tasks to each worker. This policy is suitable for scenarios in which each worker requires almost the same amount of time to process a task. WorkerLoad optimal strategy: The master node automatically detects the loads of workers. This policy is suitable for scenarios in which a large difference exists between the amount of time each worker requires to process a task. Note This parameter is applicable to agents V1.10.14 and later.
   Distribution rate	Visual MapReduce MapReduce	The task distribution rate. You can specify the number of tasks to be distributed per second or per minute.
   Number of subtasks pulled per time	Visual MapReduce MapReduce Shard run	The number of tasks that a slave node pulls from the master node at a time. This parameter is displayed only if the Subtask distribution method parameter is set to Pull model.	5
   Subtask queue capacity	Visual MapReduce MapReduce Shard run	The size of the queue that caches tasks on a slave node. This parameter is displayed only if the Subtask distribution method parameter is set to Pull model.	10
   Global concurrency of subtasks	Visual MapReduce MapReduce Shard run	The total number of concurrent tasks on all workers in the pull model. This parameter helps you limit the number of concurrent tasks. This parameter is displayed only if the Subtask distribution method parameter is set to Pull model.	1000

4. In the Timing Configuration step, configure the scheduling and advanced settings, and then click Next Step.

   

   The following table describes the timing parameters.

   Configuration item	Description
   Time Type	none: The job is not scheduled to run automatically. It is typically triggered by a workflow. cron: A cron expression. api: An API trigger. fixed_rate: A fixed interval. second_delay: A fixed delay in seconds. onetime: A one-time task. unix cron: A Unix-style cron expression.
   Cron Expression (For the cron time type only)	Enter a cron expression that complies with the cron syntax. You can also use the tool to generate and verify the expression.
   Fixed Frequency (for the fixed_rate time type only)	Enter the scheduling interval in seconds. The value must be greater than 60. For example, if you enter 200, the job runs every 200 seconds.
   Fixed Delay (Applies only to the second_delay time type)	Enter a fixed delay from 1 to 60 seconds. For example, a value of 5 triggers the job after a 5-second delay.
   Scheduling Time (For the onetime time type only)	Specify the execution time for the task. Example: 2023-7-13 00:00:00.
   UNIX Cron Expression (For the UNIX Cron time type only)	Enter a cron expression in the UNIX format.

   The following table describes the advanced configuration parameters.

   Configuration item	Description
   Time Offset	The offset between the data timestamp and the schedule time. You can retrieve this value from the context during scheduling.
   Time Zone	Select a time zone for a country or region. You can also select a standard GMT time zone.

5. In the Notification Configuration step, configure the alert settings and contacts, and then click Complete.

   

   You can set the Notification Channel and Contact Person parameter to Application Group Contact or User-defined.

   • Application Group Contact

     All contacts in the application group receive notifications.

   • User-defined

     For custom notifications, you can select a Contact Group or a Contact.

     • Contact Group

       All contacts in the contact group receive notifications. For more information about how to create a contact group, see Notification Contacts.

     • Contact

       1. Add one or more contacts to receive notifications. Click Add Contact and then click Go To Create Contact.

       

       2. On the page that appears, click New Contact and enter the contact information.

       

       

6. You can return to the Task Management page and verify that the job you created is listed and its parameters match the configuration.

   Note

   By default, a scheduled job is enabled after it is created. You can also disable or enable the job as needed. For more information, see Enable and disable scheduling tasks.

Edit a scheduling task

1. On the Task Management page, in the Operation column, click Edit for the target job.

   

2. In the Edit panel, you can configure the basic, scheduling, and alerting settings.

   Note

   You cannot change the Application ID or Task Type. The other parameters are configured the same way as when creating a task.

Run a scheduling task

On the Job Management page, you can click Run Once in the Actions column to run a scheduled job once.

Enable and disable scheduling tasks

Copy a scheduling task

On the Task Management page, click More > Copy in the Actions column for the target task. You can then edit the copied task configuration to create a new scheduling task.

Rerun a scheduling task

If your business changes, you may need to rerun a task for a past period. For example, you may need to reprocess data after you add a field to a database or find an error in the data from the previous month.

If the data from a previous task execution is incorrect or missing, you can reset the execution parameters to rerun the task and retrieve the correct data.

1. Log on to the MSE SchedulerX console.

2. On the Task Management page, in the Actions column for the target job, click More > Rebrush Task.

3. In the Rebrush Task panel, specify the Start and End Date and Data Time parameters and click OK.

   • Start and End Date: Specifies the date range for the rerun.

   • Data Time: Specifies the time for the backfill within the selected date range.

   

   Example configuration:

   • The current time is 10:00:00 on January 1, 2019.

   • The rerun date range is from October 1, 2018 to October 7, 2018. The default start time is 00:00:00 on October 1, 2018 and the default end time is 23:59:59 on October 7, 2018.

   • The data timestamp for the rerun is 11:11:11.

   The task is rerun seven times, and seven instances are generated.

   S/N	Scheduling time	Data timestamp
   1	2019.1.1 10:00:00	2018.10.1 11:11:11
   2	2019.1.1 10:00:00	2018.10.2 11:11:11
   3	2019.1.1 10:00:00	2018.10.3 11:11:11
   4	2019.1.1 10:00:00	2018.10.4 11:11:11
   5	2019.1.1 10:00:00	2018.10.5 11:11:11
   6	2019.1.1 10:00:00	2018.10.6 11:11:11
   7	2019.1.1 10:00:00	2018.10.7 11:11:11

View task execution and operation records

• To view execution records: On the Task Management page, in the Actions column of the target task, click More > History.

• View operation records: On the Task Management page, click More > Operation Record in the Actions column for the desired task.

Delete a scheduling task

1. On the Job Management page, in the Actions column for the target job, click More > Delete.

   

2. In the confirmation dialog box, click Confirm.

Import or export tasks

1. On the Task Management page, select a Professional Edition application.

   

2. Click Import Task or Export Tasks.