Dataphin:Import a deployment package

Prerequisites

• You must have the cross-tenant deployment user role and the required permissions for the object types being deployed. For more information about permissions, see Permissions for deployment objects.

• You must enable cross-tenant deployment before you can import a deployment package. For more information, see Cross-tenant deployment settings.

• You have an exported deployment package from the source environment. For more information, see Export a deployment package.

• Before importing the deployment package, configure cross-tenant deployment credentials in the destination environment. For instructions, see Cross-tenant deployment settings.

Permissions

Users with the cross-tenant deployment user role can import deployment packages.

Limitations

Manually configured data lineage in tasks is not recognized if the deployment package was exported from Dataphin 3.11 or earlier. To include this lineage data, upgrade to Dataphin 3.12 or later, re-export the deployment package, and then import it into the destination environment.

Deployment packages from single-engine tenants can be imported only into single-engine tenants. Similarly, deployment packages from multi-engine tenants can be imported only into multi-engine tenants.

Validation items

The system performs the following validation checks when you import a deployment package.

After a successful import, the system adds the package contents to the list of objects to be deployed and marks the change type for each object. For more information, see Objects to be deployed.

Procedure

1. Log on to Dataphin as a cross-tenant deployment user.

2. In the top navigation bar of the Dataphin console, choose Management Center > Cross-Tenant Deployment.

3. In the left-side navigation pane, choose Cross-Tenant Deployment > Import Package.

4. On the Import Package page, click Import Package.

5. On the Import Package page, configure the following parameters.

   Parameter	Description
   Deployment file source	You can select Local File or OSS. The OSS name corresponds to the Display Name configured in the cross-tenant deployment settings. Note To import a deployment package from OSS, you must enable OSS storage in the cross-tenant deployment settings. For more information, see Cross-tenant deployment settings.
   Deployment file	If Deployment file source is set to Local File, click the upload icon to select the downloaded deployment package, or drag the file into the upload box. If Deployment file source is set to OSS, click the browse icon, and in the Select Deployment File dialog box, select the file from the OSS directory. Important Deployment package filenames can contain only numbers (0-9), letters (a-z, A-Z), Chinese characters, and the special characters - _ . *.
   Import description	Enter a description for the import.
   Import check	No configuration is required. The system displays the file name, creation time, and the results of validation checks. For more information about the checks, see Import validation items.

6. Click Upload Deployment Package to upload the file.

7. On the Deployment Package Import Settings tab, configure the Import Policy and replacement rules.

   • Import Policy

     Parameter	Description
     General
     Owner of new objects	Do not modify: If the object's owner from the source environment exists in the destination environment, the owner is retained upon deployment. If the owner does not exist, you can assign the user performing the deployment or a specified user as the new owner. This is suitable for scenarios where the members of the source and destination environments are mostly the same. Important For objects within a project, the deployment may fail if the owner is not a member of the object's project. Change all to: Changes the owner of all objects to the user performing the deployment or a specified user. The previous owner assignment is disregarded.
     Development
     Development objects	You can deploy objects to the development environment or the production environment. Deploy to Development (Commit): Required. Commits objects in Dev-Prod projects to the development environment and objects in Basic mode projects to the production environment for intra-tenant deployments. Deploy to Production: Optional. Deploys objects in Dev-Prod projects to the production environment for intra-tenant deployments. If the commit to the development environment fails, the objects are not deployed to production.
     Tag
     Tag objects	Defaults to Deploy to Project and cannot be changed.
     Standard The data standard feature must be enabled for these settings.
     New and modified standards	For new or modified data standard objects, you can choose to Deploy as Draft or Revision and Submit for Go-live. Deploy as Draft or Revision: Required. If the change type of a data standard object is New, a draft standard is created. If the change type of a data standard object is Update, the action depends on the status of the standard in the destination environment. If the object is in a draft or under revision status, the existing standard is overwritten. If the object is published (effective, pending, or expired), a new standard is created with an under revision status based on the imported content. Submit for Go-live: Optional. This option automatically submits successfully deployed draft or revised standards for go-live, creating an approval task based on the standard approval settings configured for cross-tenant deployment. Note If an approval task is required, the deployment will fail if the task cannot be created. Only standards in a production state support mappings. We recommend deploying the data standards first, then importing the mappings.
     Standard mapping rule	For the standard mapping rule's effective status in the destination environment, you can choose to Keep the status in the current environment and update only the rule configuration (uses the status from the destination environment) or Overwrite the configuration and status of the rule in the current environment (uses the status from the source environment).
     Mapping	Select a handling policy for imported mappings in the destination environment: Overwrite or Append. Overwrite: Clears all existing mappings in the destination environment and adds the mappings from the source environment. This is suitable for scenarios where both environments must be identical. If mapping rules are executed, the mapping results may be updated. Append: Keeps existing mappings in the destination environment and adds new ones. This may cause inconsistencies if mappings deleted in the source environment are not removed from the destination environment. This is suitable for incremental updates.
     Valid mapping conflict handling	This can be configured when the Append policy is selected for mappings. If an imported mapping that is valid in the source environment is considered invalid in the destination environment, you can choose to Change "Invalid Mapping" to "Valid Mapping" or Keep "Invalid Mapping" and skip update. Change "Invalid Mapping" to "Valid Mapping": Changes the invalid mapping in the destination environment to a valid one. The deployment will fail if the invalid mapping cannot be deleted. This is suitable for scenarios where the source environment's configuration takes precedence. Keep "Invalid Mapping" and skip update: Retains the invalid mapping in the destination environment. This is suitable for scenarios where the destination environment's configuration takes precedence.
     Invalid mapping conflict handling	This can be configured when the Append policy is selected for mappings. When an imported invalid mapping is considered a valid mapping in the destination environment, you can choose to Change "Valid Mapping" to "Invalid Mapping" or Keep "Valid Mapping" and skip update. Change "Valid Mapping" to "Invalid Mapping": Changes the valid mapping in the destination environment to an invalid one. The deployment will fail if the valid mapping cannot be deleted. This is suitable for scenarios where the source environment's configuration takes precedence. Keep "Valid Mapping" and skip update: Retains the valid mapping in the destination environment. This is suitable for scenarios where the destination environment's configuration takes precedence.
     Duplicate morpheme name handling	Morphemes are uniquely identified by their names. When an imported morpheme name already exists in the destination environment, you can choose to Overwrite if duplicate or Skip if duplicate. Overwrite if duplicate: Overwrites the morpheme in the destination environment with the one from the source environment. Skip if duplicate: Retains the morpheme in the destination environment and skips the update.
     Quality The data quality feature must be enabled.
     Monitoring object content import	Supports Append and Overwrite policies. Append: Deploys only new and modified rules and schedules (conflicting names are automatically renamed) but does not propagate deletions. It also does not overwrite archive table settings or view permissions. This is suitable for incremental updates. Overwrite: Clears all existing rules and schedules for the monitoring object in the destination environment and adds those from the source environment. It also overwrites archive table settings and view permissions. This is suitable for scenarios where both environments must be identical.
     Effective status setting	Supports Keep Current Environment and Overwrite Current Environment. Keep Current Environment: Retains the effective status of the monitoring objects and quality rules in the destination environment. Overwrite Current Environment: Overwrites the effective status of the monitoring objects and quality rules in the destination environment with the status from the deployment package.
     Alert import	Supports Keep Current Environment and Overwrite Current Environment. Keep Current Environment: Retains the existing alert recipients and on-call schedules in the destination environment. This is suitable for scenarios where the destination environment has its own alert configurations. Overwrite Current Environment: Overwrites the alert recipients and on-call schedules in the destination environment. This is suitable for scenarios where both environments must be identical.
     Dependency data not found	If an imported object's dependencies do not exist in the destination environment, choose to force the deployment or report an error. Force Deploy: Ignores missing dependencies and deploys the object to the destination environment. You must edit the object to resolve the dependencies before it can be validated. Deployment Error: If a dependency is missing, the deployment of the monitoring object fails. You must resolve the dependency issues before you can deploy it.
     Security The data security feature must be enabled.
     New key	For new keys, you can choose how the key value is handled: Import Original Value, System-generated, or Automatically refresh system-generated keys, and manually update user-generated keys after import. Import Original Value: Imports the key value from the source environment for direct use in the destination environment. This is suitable for scenarios where both environments share the same consumers. System-generated: Automatically regenerates key values for all new keys. Automatically refresh system-generated keys, and manually update user-generated keys after import: Automatically refreshes the values for system-generated keys. Manually generated keys must be updated after import.
     Existing key	For keys that already exist in the destination environment, the existing key value is retained and not updated.
     Recognition result	You can filter which recognition results are imported: Import only manually specified results or Import all recognition results. Import only manually specified results: Imports only manually specified recognition results from the source environment, including those from Excel bulk uploads and those added manually. Import all recognition results: Imports all recognition results exported from the source environment.
     Manual recognition result conflict	When a manual recognition result from the source environment conflicts with a manual result in the destination environment, choose a handling policy: Keep Current Environment or Overwrite Current Environment. Keep Current Environment: Ignores the manual recognition result from the source environment and does not update it. Overwrite Current Environment: Deletes the conflicting manual recognition result in the destination environment and then adds a new one that matches the result from the source environment. Note If the source environment has a manual result and the destination environment has an automatic result, the system arbitrates to determine the final effective result after import.
     Automatic recognition result conflict	When an automatic recognition result from the source environment conflicts with an automatic result in the destination environment, choose a handling policy: Append, Overwrite, or Overwrite and Lock. Append: Appends the automatic recognition result from the source environment as a new record in the destination environment. The system then arbitrates to determine the final recognition result. The final results in the source and destination environments may not be consistent. Overwrite: Deletes all recognition results in the destination environment and adds a new automatic result that is identical to the one from the source environment. This result is not locked and may be updated by subsequent automatic recognition runs. Overwrite and Lock: Deletes all recognition results in the destination environment, adds a new automatic result identical to the one from the source environment, and locks it. A locked result cannot be changed by subsequent automatic recognition runs. Note If the source environment has an automatic result and the destination environment has a manual result, the system arbitrates to determine the final effective result after import.

   • Replacement rules

     • Data source

       Data source properties often differ between source and destination environments. Use replacement rules to bulk-update these properties before deployment. When a data source object is deployed, the first matching rule in the list is applied.

       1. Click + New Replacement Rule to add a new rule.

          Note

          • You can set a maximum of 100 replacement rules.

          • Replacement rules apply only to data sources with a change type of New or Update.

       2. In the Set Replacement Rule dialog box, configure the rule.

          Configuration step	Description
          1. Select the scope of data sources for replacement	Set data source type. Select the type of data source whose properties you want to replace, such as MaxCompute. Add scoping rules. Click + Add Rule and configure your replacement rules. Multiple rules are combined with an And operator. Configure scope. Set rule items. Available items vary by data source type. For example, MaxCompute supports scoping by Data Source Name, Owner, Endpoint, Project Name, and Access ID. Set match condition. Supported conditions include Exact Match, Does Not Match, Contains, Does Not Contain, Starts With, Ends With, Is Empty, Is Not Empty, Empty String, and Not Empty String. Note The Owner item only supports the Belongs To condition.
          2. Set replacement items and values	Configure replacement items for the production and development environments. Click New item replacement rule. Configure the replacement rule. Available replacement items vary by data source type. For example, MaxCompute supports replacing Endpoint, Project Name, Access ID, and Access key. Different data source types support different replaceable configuration items. The actual options may vary. For example, MaxCompute supports Endpoint, Project Name, Access ID, and Access key. Set match rule. Matching methods include Full text, Text Match, and Regular Expression. Matching is Case-sensitive. Note Regular expressions follow the Java specification. For more information, see the Java regular expression guide. Set replacement text. Enter the text to use as the replacement.

       3. Click Save to complete the data source replacement rule configuration.

          After you configure data source replacement rules, you can view the details of matching data sources and map data sources of the same name and type when you publish a data source object. For more information, see Map existing data sources.

          Important

          The rules are applied sequentially from top to bottom.

     • Scheduling resource group

       Lists the projects in the deployment package (excluding deleted ones). You can set the scheduling resource group for each project after import.

       Note

       • Scheduling resource group replacement is supported only on Dataphin instances deployed on the latest architecture.

       • If a task's project is not included in the deployment package, it will still be listed here. This usually means the task's scheduling resource group is not included in the destination project, and you need to specify a replacement rule.

       • If a project in the deployment package uses the tenant's default resource group, it cannot be changed. After import, it will continue to use the tenant's default resource group.

     • Offline physical table

       When deploying an offline physical table, you can replace parts of its DDL, such as the location URL of an external table. The system copies all replacement rules from the most recent package import, regardless of their status. Copied rules are disabled by default.

       1. Click + New Replacement Rule to open the New Replacement Rule dialog box.

       2. In the New Replacement Rule dialog box, select the scope of tables for replacement and set replacement items and values.

          • Select the scope of offline physical tables for replacement (Optional)

            1. Click + Add Rule and select a Configuration Item. You can choose Project Name or Table Name.

            2. Select an Operator. If the item is Project Name, the only operator is Belongs To. If the item is Table Name, operators can be Starts With, Ends With, or Regex Match.

            3. Enter or select a value. For Project Name, you can select one or more values. For Table Name, you must enter a value.

            Note

            • If no scope rules are set, the replacement applies to all offline physical tables.

            • Multiple rules are combined with an unchangeable And operator.

            • If an offline physical table in the package matches the combined rules, the replacement rule is applied.

          • Set replacement items and values

            1. Click New Item Replacement Rule to add a new blank rule.

            2. Select a Replacement Item for the rule. You can choose Entire DDL Statement or Location URL.

               • Entire DDL Statement: Replaces the entire DDL change statement for the offline physical table.

               • Location URL: Replaces the location clause within the offline physical table's DDL.

                 Note

                 • In create external table table_name (...) location '{hdfs://...}', the location URL is {hdfs://...}.

                 • In alter table table_name set location '{hdfs://...}', the location URL is {hdfs://...}.

            3. Configure the Match Rule. Select Text Match or Regular Expression, and enter the corresponding text or expression.

            4. Enter the New Item. Provide the replacement value based on the match rule. The maximum length is 512 characters.

               If the match rule is Regular Expression, the replacement value can be a regex replacement expression that supports backreferences.

       3. Click Preview Replacement. In the Preview Replacement dialog box, enter the original DDL and click Replace. The system will execute the rule and show the replaced DDL.

       Note

       Each import copies all offline physical table replacement rules from the most recent package import, regardless of their status. Copied rules are disabled by default.

     • Database SQL task

       Use replacement rules to bulk-replace the schema in database SQL tasks for the destination environment. When a database SQL object is deployed, the first matching rule in the list is applied.

       1. Click + New Replacement Rule to add a new rule.

          Note

          • You can set a maximum of 100 replacement rules.

          • Replacement rules apply only to database SQL tasks with a change type of New or Update.

       2. In the New Replacement Rule dialog box, configure the rule.

          Configuration step	Description
          1. Select the scope of data sources for replacement	Data source type Select the data source type corresponding to the database SQL task, such as PostgreSQL. Add rule Click + Add Rule and configure your replacement rules. Multiple rules are combined with an And operator. Data Source Name: Conditions include Exact Match, Does Not Match, Contains, Does Not Contain, Starts With, Ends With, Is Empty, Is Not Empty, Empty String, and Not Empty String. You must also enter a string value. Project name: The only condition is Belongs To. You must select a destination project from the list of all projects in the current tenant. Schema: The only condition is Exact Match. You must enter a specific schema name.
          2. Set replacement items and values	Configure replacement items for the Production environment and Development environment separately. The configuration for the development environment is the same as for production. Click New item replacement rule. Configure the replacement rule in the new configuration item. Replace Configuration Item supports only schema, and Matching Rule supports only Full Text. You must also enter a string for New Configuration Item. (Optional) You can select multiple replacement rules in the production environment and click Copy to Development Environment.

       3. Click Save to save the database SQL task replacement rule.

          Important

          The rules are applied sequentially from top to bottom.

8. Click Save or Save and Deploy to start deploying the objects.

Next steps

After importing the deployment package, go to the package overview page to view and manage the deployment objects. For more information, see Objects to be deployed.