Create DataWorks jobs - SchedulerX - Alibaba Cloud Documentation Center

SchedulerX allows you to create DataWorks jobs and orchestrate DataWorks jobs with other jobs in SchedulerX to process data.

Prerequisites

You have logged on to the DataWorks console and have activated DataWorks Enterprise Edition.

Client access procedure

Use SDK dependencies

Use the dependencies of the SchedulerX SDK client.

Upgrade the SchedulerX client to V1.3.4 or later.
When you connect the client to DataWorks Enterprise Edition, write the AccessKey ID and AccessKey secret of your Alibaba Cloud account to the startup configuration. Make sure that your Alibaba Cloud account has operation permissions on DataWorks Enterprise Edition.
```
# Account information that is used to access DataWorks.
spring.schedulerx2.aliyunAccessKey=AccessKey ID of the Alibaba Cloud account
spring.schedulerx2.aliyunSecretKey=AccessKey secret of the Alibaba Cloud account
```

Use an agent

If you use an agent for access, deploy the agent and use the agent to access DataWorks by following the instructions in Use the SchedulerX agent to connect an application to SchedulerX (Script or HTTP jobs). The following items are required:

Installation package. Click the link to download it.
Add the AccessKey ID and AccessKey secret of the Alibaba Cloud account that has permissions to operate DataWorks to the startup parameters in the agent.properties file.
```
# Account information that is used to access DataWorks.
aliyunAccessKey={ALIYUN_ACCESS_KEY}
aliyunSecretKey={ALIYUN_SECRET_KEY}
```

Docker images

Select a Docker image based on your network and CPU architecture.

Network	x86_64	arm64
Internet	registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:1.10.13-dataworks-amd64	registry.cn-hangzhou.aliyuncs.com/schedulerx/agent:1.10.13-dataworks-arm64
Virtual private cloud (VPC) in China (Hangzhou)	registry-vpc.cn-hangzhou.aliyuncs.com/schedulerx/agent:1.10.13-dataworks-amd64	registry-vpc.cn-hangzhou.aliyuncs.com/schedulerx/agent:1.10.13-dataworks-arm64

If you use a Docker image to start, configure the startup environment variables ALIYUN_ACCESS_KEY and ALIYUN_SECRET_KEY.

Create a DataWorks job

Perform the following operations in the DataWorks console:
1. Create a manually triggered workflow. For more information, see Create a manually triggered workflow.
2. Create a node without configuring dependencies. For more information, see Create nodes and configure node dependencies.
3. Submit the workflow. For more information, see Commit a workflow.
Perform the following operations in the SchedulerX console:
1. Create a workflow and add the DataWorks nodes (jobs) to the workflow. For more information, see Create a workflow.
2. After the job is added, drag and drop the points of the two ends of the job to the points of another job to create a dependency relationship. This way, a complete workflow is created.
3. Configure scheduled triggering for the created workflow. For more information, see Cron.

Results

After the workflow is triggered, you can view the progress of the workflow and the status of each node in the workflow instance list. You can right-click a node to view details about the execution result of the node.

What to do next

In the SchedulerX job instance list, you can view the execution details of a job node, terminate a job, or rerun a job.
In DataWorks Operation Center, you can query information about the DataWorks node instances that are scheduled.