Manage and publish global objects for cross-tenant publishing after importing a deployment package - Dataphin - Alibaba Cloud - Dataphin

Prerequisites

You have imported a deployment package from the source environment into the destination environment. For instructions, see Import a deployment package.

Access global objects

Log in to Dataphin with an account that has cross-tenant deployment permissions.
From the top navigation bar on the Dataphin homepage, choose Admin Center > Cross-tenant Deployment.
In the left-side navigation pane, choose Cross-tenant Deployment > Import Deployment Package.
In the Actions column for the target deployment package, click the Deploy icon .
Click the Global tab to view the global objects pending deployment.

View and manage global objects

After you import a deployment package, you can perform operations on the Global tab, such as filtering, deploying, viewing details, comparing versions, and viewing version details for scheduling cycles, global variables, common calendars, data domains, compute engines, projects, data sources, object attributes, and identification features. The available operations may vary slightly for different types of objects.

Actions	Description
Filter	On the Global tab, you can filter each type of object. The available filter options vary by object type. Batch Search: All object types support batch search. Click Batch Search. In the Batch Search dialog box, enter the search terms for the objects. You can enter up to 1,000 items, separated by line breaks. After you click OK, the system runs an exact match search and displays the results. Filter: Deployment Status: Filter by status, including Pending Deployment, Queued, Deploying, Deployment Succeeded, Deployment Failed, and Succeeded with Risks. Queued indicates that the object is in the deployment queue and is waiting to be deployed. Succeeded with Risks indicates that the deployment was successful, but the deployed content is not identical to the imported content. This discrepancy does not affect the object's usability but flags a potential risk. Change Type: Filter by the type of change, including Add, Update, and Delete. For more information about change types, see Change type descriptions. Environment Change: Filter by whether the destination environment has changed since the last import. Compute Engine Type: Only compute engine objects support filtering by compute engine type. You can select one type at a time, such as MaxCompute, Blink, Ververica Flink, Hologres, Flink, or FusionInsight Flink. Data Domain: Only project objects support filtering by data domain. You can select one or more data domains. You can also search for a specific data domain by entering keywords. Data Source Type: Only data source objects support filtering by data source type and by same name and type. You can select one or more data source types. You can also search for a specific data source by entering keywords. For a list of supported data sources, see Dataphin-supported data sources. Same Name and Type: Only newly added data source objects support this filter. This filter checks if a data source with the same name and type already exists in the destination environment. Attribute Type: Only object attribute objects support filtering by attribute type and modifier. This includes quality rule attributes. You can select one or more attribute types. Modifier: Only object attribute objects support filtering by attribute type and modifier. You can select one or more recent committers provided by the system. Modification Time: Only object attribute and identification feature objects support filtering by modification time. You must specify a start and end date in the `YYYY-MM-DD hh:mm:ss` format.
Deploy	You can redeploy objects that failed deployment or are pending deployment. You can deploy objects individually or in batches. Successfully deployed objects do not need to be redeployed. Single deployment: In the Actions column for the target object, click Deploy. In the dialog box, enter a name for the deployment (up to 64 characters) and click OK. Note When you deploy a Flink compute engine individually, you must re-enter the compute engine information on the edit page. Batch publish: Select the checkboxes for the target items, or click Select All or Select All on This Page at the bottom of the page. After making your selection, click Publish at the bottom of the page. In the Publish dialog box, enter a name of up to 64 characters and click OK. Note When you batch-deploy data sources, any data source with a duplicate name is automatically ignored. You must first map these data sources or deploy them individually. When you deploy a data source, the system checks its connectivity. If you batch-deploy multiple data sources and some do not support concurrent operations, the process might be slow. Data source types that do not support concurrent operations include SAP Table, HDFS, Hive, Impala, Kudu, Kafka, and other types that support Kerberos. Select All Results applies the operation to all objects that match the current filter and search criteria. Select Current Page applies the operation only to the objects on the current page. Before you deploy a project, ensure its dependent compute engine objects are already deployed. Otherwise, the deployment will fail. Compute engine objects do not support batch deployment.
Deployment details	View the details of a single deployment, including the Deployment Status, the object's Basic Information, the Validation Process results, and any Warnings and Errors. The validation process for object attribute and identification feature objects includes a Permission Check, Duplicate Name Check, and Deployment Validation. For all other object types, the validation process only includes Deployment Validation. Permission Check: If the permission check setting for cross-tenant deployment is set to Validate Operator Permissions, the system validates the operational permissions for each object in the destination environment during import. If the setting is Skip Permission Validation, this check is skipped. Note During deployment, the system checks whether the member, after being replaced according to the import policy, has the required operational permissions. Duplicate Name Check: When the change type for an object attribute is Add or Update, the system checks if the name of the quality rule attribute is already in use. Deployment Validation: Checks if the deployment process can succeed. In addition to re-running the pre-checks, it performs the following: if the change type is Add or Update for an object attribute, it validates the number of quality rule attributes; if the change type is Delete, it verifies that the object has been deleted. Note If an object's change type is Delete and its deployment status is Succeeded, the object no longer exists in the destination environment, and you cannot view its deployment details.
Mapping	You can map data sources individually or in batches. If data sources with the same name and type exist, you can configure batch mapping settings. For instructions, see Map an existing data source. Note Only data source objects support the mapping operation. For data sources with the same name and type, you can either map them or rename them before deployment. For data sources with the same name but a different type, you must rename them before deployment.
Version details	View the version information for the current data source object, including its basic information, production data source, and development data source. Note Only data source objects support viewing version details.
Version comparison	You can view a version comparison only when the change type is Update. This compares the imported object version (from the source environment's deployment package) with the latest version of the corresponding object in the destination environment. Note Only data source, object attribute, and identification feature objects support viewing a version comparison.
Set replacement rules	You can set replacement rules for individual or multiple data sources. To batch-update data source properties, you must first set up replacement rules. After setting the rules, you can use them to batch-deploy the affected data sources. For instructions, see Set data source replacement rules. Note Only data source objects support setting replacement rules. Replacement rules are applied during data source deployment. When you deploy a single data source, you can choose to enable or disable the replacement rules. To batch-update data source attributes, you must first set up replacement rules. You can then use the rules to perform a batch deployment.

Set data source replacement rules

Note

Only data source objects support this feature.

The attribute configurations of the source and destination environments often differ. Before deployment, you can use replacement rules to batch-update the data source configurations to match the destination environment. When deploying a data source, the system applies the first matching rule to replace its attributes.

Below the list of data source objects, click Set Replacement Rules.

Edit an existing rule or create a new one.

To edit a rule: In the Actions column for the target rule, click the Edit icon to open the New Replacement Rule dialog box.
To create a new rule: Click + New Replacement Rule to open the New Replacement Rule dialog box.

Parameter

Description

Scope of data sources for replacement

Set data source type.

Select the data source type for which you want to replace attributes, such as MaxCompute.
Add scoping rules.

Click + Add Rule and configure the rule conditions. The conditions are joined by an AND operator.
Configure selection scope.
1. Set rule conditions.
  
  The available conditions depend on the data source type. For example, a MaxCompute data source supports scoping by Data Source Name, Owner, Endpoint, Project Name, and Access ID.
2. Set match conditions.
  
  Supported conditions include Exact Match, Does Not Match, Contains, Does Not Contain, Starts With, Ends With, Is Empty, Is Not Empty, Empty String, and Non-empty String.
  
  Note
  For the Owner condition, the only supported match type is Is one of.

Set replacement attributes and values

Configure replacement attributes for the production and development environments.

The available attributes depend on the data source type. For example, a MaxCompute data source supports replacing Endpoint, Project Name, Access ID, and Access key.
Set matching rule.

Match rules include Full text, Text match, and Regular expression. The matching is case-sensitive.

Note
Regular expressions follow the Java language specification. For more information, see the Java regular expression guide.
Set replacement text.

Enter the text to use as the replacement.

After you complete the configuration, the rules are applied in order to perform the replacements.

Click Save.

After you save the rule, the system identifies the data source objects that match the rule and displays the count.

In the Actions column for the target rule, click the Deploy icon to batch-deploy the data sources according to the rule.
Click to view the data sources that match the rule.

In the Data Sources Matched by Rule dialog box, you can view details, deploy, and map data sources that have the same name and type.

Map an existing data source

Note

Only data source objects support this feature.

If an imported data source has the same name and type as an existing one in the destination environment, you can configure a mapping. In the Map Existing Data Source dialog box, click View Details for Comparison to see the differences between the imported data source and the existing one.

You can choose to map with or without updating the configuration.

Map and Update Configuration: Maps the imported data source to the existing one and updates the destination data source's configuration to match.

Note
If a replacement rule is matched, the data source configuration is updated according to that rule.
Map Only, Do Not Modify Configuration: Maps the imported data source to the existing one but does not update the configuration of the destination data source.

Single tenant with multiple engines

In a multi-engine environment, if a compute engine you deploy is configured to be Configure Separately, you can deploy it directly, even if the multi-engine tenant does not support this configuration method. If the configuration method is Reference Cluster but the destination tenant does not have the corresponding cluster, you can click Configure Cluster to go to the cluster management page and create the required cluster.