Manually triggered workflows

Usage notes

• Manually triggered workflows run on demand only. Automatic scheduling is not supported.

• Lines between nodes indicate execution order, not scheduling dependencies.

• Manually triggered workflows differ from scheduled Workflows.

Design a manually triggered workflow

1. Create a workflow

   1. Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose Shortcuts > Data Studio in the Actions column.

   2. In the left-side navigation pane, click . To the right of Manually Triggered Workflow, click > Create Manually Triggered Workflow. Specify a name for the workflow and click Confirm to open the workflow editor page.

2. Orchestrate nodes

   1. At the top of the workflow editor, click New Internal Node. Select a Type for the internal node, specify a Name, and click Confirm. The node appears on the canvas.

   2. Plan the workflow as needed.

      A large number of nodes can degrade runtime performance. Keep workflows under 100 nodes for optimal performance. The maximum is 200 nodes per workflow.

3. Configure workflow parameters

   To share parameters among all nodes in a workflow, go to the Scheduling Settings > Scheduling Parameters pane on the right side of the canvas and click Add parameters. You can then reference the parameter in your node code by using the ${Parameter name} format.

   Note

   In manually triggered workflows, workflow-level parameters take precedence over scheduling parameters with the same name on internal nodes. This differs from scheduled workflows.

4. Priority and concurrency (advanced configuration)

   When multiple workflows or tasks are triggered simultaneously and cause resource contention, use Priority and Weighting Strategy to ensure critical tasks run first.

   • Ensure core business priority: Set higher priority for core business workflows so they run before non-core workflows.

   • Reduce critical workflow duration: Within the same workflow instance, you can use the Priority weighting strategy to influence node execution order. The Downstream Weighting strategy assigns higher weight to nodes with more upstream dependencies, prioritizing critical-path execution and reducing overall duration.

     Configuration item	Description
     Priority	Sets the priority level of a workflow instance in the scheduling queue. Available levels: 1, 3, 5, 7, and 8 (higher number = higher priority). Higher-priority tasks always acquire scheduling resources first.
     Priority weighting strategy	Determines how node weights are calculated within a workflow at the same priority level. Nodes with higher weights run first. No weighting: All nodes have a fixed base weight. Downstream weighting: Weights are adjusted dynamically based on upstream dependencies. Nodes with more upstream dependencies get higher weights, prioritizing critical-path execution. Weight = Initial weight value + sum of priorities of all upstream nodes.
     Max Parallel Instances	Sets the maximum concurrent instances of this workflow. When the limit is reached, new instances enter a pending state. Set to Allowed or a custom value up to 100,000. Note If the configured limit exceeds the resource group capacity, the actual concurrency is capped by the resource group.

     The DataWorks priority system follows a hierarchical override rule: Runtime specification > Node-level configuration > Workflow-level configuration.

     1. Workflow-level (baseline): Set in the Scheduling Policy of the workflow. Applies as the default for all nodes.

     2. Node-level (override): In the Scheduling Settings > Scheduling Policy of a specific node, set a higher Priority for that node to override the workflow-level setting.

     3. Runtime (temporary): Use the Operation and Maintenance Center when you manually run the workflow, via the Runtime Priority Reset toggle. Takes effect for the current run only and does not modify permanent settings.

Develop business logic

DataWorks abstracts engine capabilities into node types, eliminating the need for complex command lines. Use general-purpose node types for complex logic processing.

Develop business logic with synchronization nodes, compute nodes, and other node types.

• Synchronization nodes transfer data between databases by configuring source and destination connections.

• Use data development nodes for tasks such as MaxCompute SQL queries. DataWorks provides a visual interface for creating resource and function nodes. Develop data. Create resources and functions.

Deploy a manually triggered workflow

In a standard mode workspace, Data Studio is for development and testing only. Deploy the entire workflow to push all nodes to production in a batch.

1. Click Run at the top of the workflow to open the run page and verify that each node runs successfully.

2. On the workflow run page, click Return at the top, and then click Publish.

3. Click Start Release Productionand select Incremental Publish or Full Publishing.

   • Incremental Publish: Deploy selected internal nodes.

   • Full Publishing: Deploy the entire workflow and all internal nodes.

4. Complete the deployment steps in sequence: Build Package, Development Checker, Deploy to Development Environment, Production Checker, and Deploy to Production Environment.

Run the workflow in production

1. After deployment, click Go to operation and maintenance on the deployment page to open Operation Center.

   Note

   Alternatively, open the DataWorks workspace list page, select the target region, find the workspace, and click Shortcuts > Data operations and maintenance in the Operation column to open the O&M page. Find Manually Triggered Task O&M in the left-side navigation pane.

2. ClickRun in the Operation column of the target task.

Other operations