This topic describes how to deploy nodes in a workspace in standard mode and how to use the cross-workspace cloning feature to clone and deploy nodes in a workspace in basic mode.

Background information

In a rigorous data development process, developers develop and debug code and configure dependencies and scheduling properties for nodes in the development environment. Then, developers commit the nodes to run them in the production environment.

DataWorks workspaces in standard mode can process data from the development environment to the production environment within a single workspace. We recommend that you use workspaces in standard mode to develop and produce data. For more information, see Basic mode and standard mode.

In a workspace in standard mode, committed nodes are automatically added to the Create Deploy Task page. This page displays created, updated, and deleted nodes, resources, and functions.

After you deploy a node on the Create Deploy Task page, a deployment task is generated for the node. You can view the deployment record and status of the node on the Deploy Tasks page.Deploy

To make the nodes, resources, and functions that you create, update, or delete on the DataStudio page take effect in the production environment, you must deploy them to the production environment on the Create Deploy Task page. On the Create Deploy Task page, you can add one or more nodes to the list of nodes to be deployed and deploy the nodes at a time.

On the Create Deploy Task page, you can modify the number of items that can be displayed on each page.

Find a node and click View. You can view the code changes of the current version.

The time when instances are generated varies based on the instance generation mode.
  • Nodes with the Start Instantiation parameter set to Next Day: If you update and deploy an auto triggered node before 23:30, instances are generated for the updated node the next day.
  • Nodes with the Start Instantiation parameter set to Immediately After Deployment: If you create and deploy a node, instances whose scheduled time is 10 minutes later than the time when the node is deployed are generated as expected. If you update and deploy a node, instances whose scheduled time is 10 minutes later than the time when the node is deployed are generated again based on the latest scheduling configuration. These instances replace those that are generated before the update. For more information, see Immediate instance generation.
  • If you create or update a node and deploy the node after 23:30, instances are generated for the new or updated node on the third day.
  • If you deploy a node after 23:30 and the Start Instantiation parameter is set to Immediately After Deployment, instances are not immediately generated for the node.

Deploy nodes in a workspace in standard mode

Each DataWorks workspace in standard mode is associated with two MaxCompute projects, one as the development environment and the other as the production environment. You can directly commit and deploy nodes from the development environment to the production environment.

  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. Find the required workspace and click Data Analytics.
  2. Commit nodes.
    1. Double-click a configured workflow. On the tab that appears, click the Submit icon in the top toolbar.
    2. In the Commit dialog box, select the nodes to be committed, set the Change description parameter, and then select Ignore I/O Inconsistency Alerts.
      Note If all nodes of a workflow are committed and you modify only the workflow or node properties, you can enter the description and commit the workflow, without the need to select the nodes. The changes are automatically committed.

      If a node has been committed and the node code remains unchanged, you cannot select the node again.

    3. Click Commit.
  3. After you commit the nodes, click Deploy in the upper-right corner.
  4. On the Create Deploy Task page, select the nodes to be deployed at a time and click Add to List.
    You can filter and search for nodes by setting parameters such as Committed By, Node ID, Node Type, and Change Type. You can click Deploy Selected to immediately deploy the selected nodes to the production environment.Add to List
  5. Click To-Be-Deployed Node List. Verify that the node information in the list is correct and click Deploy All. All nodes in the list are deployed to the production environment.
    Deploy All
    Note Workspaces in basic mode do not allow you to directly perform operations on table data in the production environment. Workspaces in standard mode ensure a stable, secure, and reliable production environment. Therefore, we recommend that you deploy and run nodes in a workspace in standard mode.

Deploy nodes in a workspace in basic mode

If you want to isolate the development environment from the production environment for workspaces in basic mode, create two workspaces, one for development and the other for production. You can clone nodes from the development workspace to the production workspace.

For example, two workspaces in basic mode are created, one for development and the other for production. You can use the cross-workspace cloning feature to clone nodes from Workspace A to Workspace B, and then commit the cloned nodes to the scheduling engine for scheduling in Workspace B.
Note
  • Permission requirement: Only workspace administrators and RAM users who are assigned the administration expert role can clone nodes. The administration expert role has permissions to create clone tasks and deploy cloned nodes.
  • Supported workspace type: You can clone nodes only from workspaces in basic mode to other workspaces.
  • Prerequisites: The source workspace in basic mode and the destination workspace in standard mode are created.
  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. Find the required workspace and click Data Analytics.
  2. Commit nodes.
    1. Double-click a configured workflow. On the tab that appears, click the Submit icon in the top toolbar.
    2. In the Commit dialog box, select the nodes to be committed, set the Change description parameter, and then select Ignore I/O Inconsistency Alerts.
    3. Click Commit.
  3. Click Cross-project cloning in the upper-right corner.
  4. On the Create Clone Task page, select the nodes to be cloned and set the Clone to target workspace parameter.
  5. Click Set Compute Engine Mapping. Configure the mapping between the compute engines of the current workspace and the destination workspace.
    If the destination workspace has multiple compute engines, you must configure the mapping between the compute engines of the current workspace and the destination workspace before you clone nodes. If no mapping is configured, the nodes are cloned to the default compute engine of the destination workspace.
    Note
    • If the type of the compute engine to which the nodes to be cloned belong does not exist in the destination workspace, a message appears in the Set Compute Engine Mapping dialog box. You can choose to skip these nodes to filter the nodes that cannot be cloned. Otherwise, an error is reported during the cloning process.
    • The Set Compute Engine Mapping button is displayed only when an engine type in the source or destination workspace has more than two engine instances.
  6. Click Add to List. The selected nodes are added to the list of nodes to be cloned.
    Clone
  7. Click To-Be-Cloned Node List in the upper-right corner. Click Clone All.
  8. After the engine mapping is prechecked, confirm the information and click Clone.
  9. After the nodes are cloned, go to the destination workspace and view the cloned nodes. Generally, the overall directory structure of the workflow is cloned.