Cross-workspace deployment

DataWorks allows you to add multiple objects, such as nodes, functions, resources, and components, to a deployment package and deploy them in a single batch. This topic describes how to create and deploy a deployment package for cross-workspace deployment.

Use cases

Cross-workspace deployment is primarily used to deploy objects such as nodes, functions, resources, and components from a workspace in basic mode to another workspace within the same region and under the same Alibaba Cloud account. You can also use this feature to isolate the development and production environments for workspaces in basic mode. For more information, see Appendix: Isolate the development and production environments in basic mode.

Note

To ensure that tasks run correctly in the destination workspace after deployment, read Deployment change logic carefully before you proceed. Ensure you understand the mapping logic between the source and destination workspaces.

Limitations

Permissions

Create a deployment environment: Only members with the Workspace Administrator role can create and configure a deployment environment.
Deploy tasks to the production environment of the source workspace: Members with the Operator, Deployer, or Workspace Administrator role can deploy tasks to the source workspace's production environment in Data Studio.
Create a deployment package: Members with the Development role can create a deployment package in Deployment Center.
Deploy a deployment package: Members with the Operator, Deployer, or Workspace Administrator role in both the source and destination workspaces can deploy a deployment package.

For more information about adding members to a workspace and granting permissions, see Manage permissions on workspace-level services.

Environment limitations

The following table describes the environment requirements for deploying objects from a source workspace to a destination workspace.

Source workspace		Destination workspace
Workspace mode	Data Studio version	basic mode New Data Studio	basic mode legacy Data Studio	standard mode New Data Studio	standard mode legacy Data Studio
basic mode	New Data Studio	Supported	Not supported	Supported	Not supported
basic mode	legacy Data Studio	Not supported	Supported	Not supported	Supported
standard mode	-	Not supported	Not supported	Not supported	Not supported

Other limitations

The Cross-workspace Deployment feature supports deploying objects across different workspaces within the same region and under the same Alibaba Cloud account. Cross-region deployment is not supported. If you need to deploy objects across regions, use Deployment Center.

Deployment process

Procedure

All operations for a cross-workspace deployment are performed in the source workspace.

Step 1: Create a deployment environment

To deploy nodes from a source workspace, you must first define the destination workspace as a deployment environment. This involves mapping compute resources, resource groups, and dependencies from the source to the destination. When you deploy the package, the system uses these mappings to automatically reconfigure the nodes for the destination environment.

Note

Only members with the Workspace Administrator role can create and configure a deployment environment.

1. Go to Deployment Center

Log on to the DataWorks console. In the target region, click More > Management Center in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Management Center.
In the upper-left corner, click the icon and choose All Products > More > Deployment Center.

2. Configure the deployment environment

On the Deployment Center page, click Deployment Environment Management in the left-side navigation pane.

Click Create Deployment Environment and configure the parameters.

Configure basic information for the environment.

Parameter	Description
Environment Name	The name of the new deployment environment.
Deployment Type	Select Cross-workspace Deployment.
Destination workspace	Select the destination workspace for the deployment.
Deployment Object Owner	The owner of each node in the destination workspace after a successful deployment. Valid values: Default: DataWorks attempts to set the original node owner as the new owner. If the original owner is not a member of the destination workspace, ownership is assigned to the user who deploys the package (the executor). Deployment Package Creator: DataWorks assigns node ownership to the user who created the deployment package. Deployment Package Executor: DataWorks assigns node ownership to the user who deploys the package.

Configure Computing Resource Mappings.

Map the compute resources between the source and destination workspaces. During deployment, the system uses these mappings to deploy objects to the corresponding compute resource in the destination workspace.

Note

In the Computing Resource Mappings section, you must configure a mapping for at least one Computing Resource in Source Workspace.
Determine whether you need to map a source compute resource based on your business requirements.

Parameter	Description
Computing Resource in Source Workspace	Displays all compute resources that are bound to the source workspace.
Computing Resource Type	The type of the compute resource bound to the source workspace.
Computing Resource in Destination Workspace	Select the destination compute resource to which you want to map the source compute resource. You can go to the Deployment Center > Deployment Environment Management page in the target workspace and click View Workspace Environment Information in the upper-right corner to view compute resource information.
Ignore Mapping	If enabled, the source compute resource is not mapped. Objects associated with this resource, such as nodes, resources, and functions, are not included in the deployment package. If you do not need to deploy the nodes, resources, functions, or components under a specific compute resource, you can turn on Ignore Mapping for that resource.

Configure Resource Group Mappings.

Map the resource groups between the source and destination workspaces. During deployment, the system uses the mapped destination resource groups to run tasks.

Note

DataWorks only supports mapping between resource groups of the same type.
When mapping a Data Integration resource group, ensure that configurations such as the IP address whitelist and network connectivity are consistent between the source and destination resource groups. Otherwise, Data Integration tasks may fail. For more information about network configuration, see Network connectivity solutions.

Parameter	Description
Resource Group in Source Workspace	Displays all resource groups that are bound to the source workspace.
Resource Group Type	The type of the resource group bound to the source workspace.
Resource Group in Destination Workspace	Select the destination resource group to which you want to map the source resource group. You can go to the Deployment Center > Deployment Environment Management page in the target workspace and click View Workspace Environment Information in the upper-right corner to view the resource group information.
Ignore Mapping	If enabled, the source resource group is not mapped, and deployed tasks continue to use it. Note The Ignore Mapping feature is not supported for default or public resource groups. If the source workspace has more resource groups than the destination workspace, you can turn on Ignore Mapping for the extra resource groups. Deployed tasks will then continue to use the source resource groups.

Configure Dependency Mappings.

This section displays all upstream projects that have cross-project dependencies in the source workspace. You can map each source project to a target project. During deployment, the system uses these mappings to point the deployed nodes to the new upstream projects in the destination workspace.

Note

By default, the Dependency Mappings section displays dependencies that are updated daily (T+1). To view real-time dependencies, click Refresh Dependencies..
If you map cross-project dependencies to new projects, DataWorks modifies node dependencies accordingly. Ensure that the mapped workspace allows new dependencies to be added. For information about common issues, see An error is reported when you commit a node: The output name of the parent node on which the current node depends does not exist.
For MaxCompute tasks, such as ODPS SQL and ODPS Spark nodes, configuring dependency mappings also updates the task code accordingly.

Parameter	Description
Source Project Name	Displays the names of all projects with cross-project dependencies in the source workspace.
Property	The project category.
Destination Project Name	Select the destination project to which you want to map the source project.
Ignore Mapping	If enabled, the dependency mapping for this project is ignored during deployment, and the node's dependency on the source project remains unchanged.

Configure Data Source Mappings.

Map the data sources between the source and destination workspaces. During deployment, tasks are updated to use the mapped destination data sources.

Parameter	Description
Source Name	Displays all data sources in the source workspace.
Data Source Type	The type of the data source.
Destination Name	Select the destination data source to which you want to map the source data source.
Ignore Mapping	If enabled, the system ignores the data source during deployment and does not map it to the destination workspace.

Configure Parameter Mappings.
Map workflow and scheduling parameters from the source workspace to the destination workspace. You can define parameter names and values by using the UI or expressions. For more information about how to configure parameters, see Formats of scheduling parameters.
Note
Ensure that parameter names are unique within each deployment to avoid assigning the same parameter value to different tasks.
Click Create to create the deployment environment.

Step 2: Deploy to source production

This example uses a workspace in basic mode that is running New Data Studio. In this step, you deploy tasks to the production environment of the current (source) workspace. After deployment, the system adds the deployed objects to the Nodes to Deploy list in Deployment Center, ready for cross-workspace deployment.

Note

Only members with the Operator, Deployer, or Workspace Administrator role can perform deployments.
For a workspace in basic mode that uses legacy Data Studio, find the node, function, or other object you want to deploy in Data Studio and click the Commit () icon in the top toolbar.

In Data Studio, find the object to deploy and click Deploy in the top toolbar.
In the deployment dialog box, click Start Deploying to Production.
The deployment process consists of three stages: Deployment package build, Production checker, and Deploy to production environment. When a stage is complete, its status changes to Completed. You can click View Details to see execution information.

Step 3: Create a deployment package

Note

Members with the Development role can create a deployment package in Deployment Center. If you have the Workspace Administrator role, you do not need to manually create a deployment package. After you select the objects to deploy, you can deploy them in a batch, and the system automatically generates a deployment package.

Log on as a user with the Development role and go to the Deployment Center of the source workspace.
From any DataWorks module, click the icon in the upper-left corner and choose All Products > More > Deployment Center.
In the left-side navigation pane, click Nodes to Deploy to view the objects that are ready for deployment. In the Actions column of an object, click Add to Deployment Package. After you add an object, the number of objects to deploy is shown on the View Deployment Package button.
Note
- You can also select multiple objects and click Batch Add to Deployment Package at the bottom of the page.
- If you use New Data Studio and click Terminate Deployment during the deployment process, the corresponding objects are removed from this list.
After adding objects to the package, click View Deployment Package to verify the package contents. You can click Remove to exclude unwanted objects from the package.
Note
Currently, you can view only objects from legacy Data Studio, not from New Data Studio.
On the Pending Deployment page, click Deployment Package Creation and follow the on-screen instructions to create the deployment package.
Note
If you have the Workspace Administrator role, the Deployment Package Creation button is not displayed. Instead, the Deploy All button appears, allowing you to deploy the objects directly.

Step 4: Deploy the deployment package

Note

You must have the Operator, Deployer, or Workspace Administrator role in both the source and destination workspaces.

Log on as a user with the Operator, Deployer, or Workspace Administrator role and go to the Deployment Center of the source workspace.
From any DataWorks module, click the icon in the upper-left corner and choose All Products > More > Deployment Center.
In the left-side navigation pane, click Deployment Packages to view the generated packages.
In the Actions column of the target package, click Deploy and follow the on-screen instructions.
Note
- During a cross-workspace deployment, users with the Operator, Deployer, or Workspace Administrator role can edit the resource group that deployed tasks will use in the destination workspace.
- For nodes that generate instances on the following day (T+1), the deployment takes effect based on the following schedule:
  - If you deploy before 23:30, the changes take effect the next day.
  - If you perform the deployment after 23:30, the changes take effect two days later.
- For nodes that generate instances immediately, the deployment takes effect based on the following schedule:
  - Instances scheduled to run more than 10 minutes after the deployment time will be generated or updated normally.
  - For instances scheduled to run within 10 minutes of the deployment time, or for any deployment after 23:30, the changes take effect two days later.

Step 5: View the deployment result

After the deployment package is created and deployed, you can view the deployment result on the Deployment Packages page of the current workspace.
On the Deployment Packages page, confirm that the Deployment Status column shows Deployment Successful. This indicates that the cross-workspace deployment is complete.
After the deployment succeeds, go to the Operation Center of the destination workspace to view the deployed tasks.