SchedulerX allows you to create DataWorks jobs and orchestrate DataWorks jobs with other jobs in SchedulerX to process data.
Prerequisites
The SchedulerX agent is updated to V1.3.4 or a later version.
The agent is connected to SchedulerX. The AccessKey ID and AccessKey secret of the Alibaba Cloud account that have permissions to access DataWorks Enterprise Edition are added to the agent.properties file. For more information, see Use the agent to connect an application to SchedulerX (Script or HTTP jobs).
# Account information that is used to access DataWorks. spring.SchedulerX2.aliyunAccessKey=The AccessKey ID of the Alibaba Cloud account spring.SchedulerX2.aliyunSecretKey=The AccessKey secret of the Alibaba Cloud account
Create a DataWorks job
Perform the following operations in the DataWorks console:
Create a manually triggered workflow. For more information, see Create a manually triggered workflow.
Create nodes. You do not need to configure node dependencies. For more information, see Create nodes and configure node dependencies.
Submit the workflow. For more information, see Commit a workflow.
Perform the following operations in the SchedulerX console:
Create a workflow and add the DataWorks nodes (jobs) to the workflow. For more information, see Create a workflow.
After you add the jobs, connect the output port of a job to the input port of another job and repeat the step to connect subsequent jobs to form a workflow.
Configure scheduling settings for the workflow. For more information, see Cron.
Result
After the workflow is triggered, you can view the progress of the workflow and the status of each node in the workflow instance list. You can right-click a node to view details about the execution result of the node.
What to do next
In the SchedulerX job instance list, you can view the execution details of a job node, terminate a job, or rerun a job.
In DataWorks Operation Center, you can query information about the DataWorks node instances that are scheduled.