Dataphin:Batch import and export Data Standards

Limits

A single data standard set can contain no more than 1000 standards.

Permission description

• Super administrators and data standard administrators can batch import data standards across all data standard sets.

• Data standard set maintenance personnel and data standard set members can batch import data standards for the sets they are part of.

• All members with view permissions can export data standards.

Batch import standards

1. On the Dataphin home page, select Administration > Standard from the top menu bar.

2. Click Standard in the left-side navigation pane. In the all data standards area of the standard list page, click Batch Import And Export , and select Batch Import Standards .

3. In the Batch Import Data Standards dialog box, configure the relevant parameters.

   Parameter	Description
   Standard Template	Please select the standard template referenced by the data standard you want to import. Different standard templates have different filling specifications. If you need to import based on multiple templates, please upload them separately.
   Template Type	Support two import templates: Add and Update. Add type template: Only includes the standard information table header that needs to be configured; you can also pre-configure part of the information to generate pre-filled content of the template to reduce filling costs. Update type template: You need to configure the range of standards to be exported. The exported file will contain all the selected data standard configuration information to be updated. You only need to modify the information that needs to be revised and then re-import it without reconfiguring everything. Note The selected template type only affects the generation of the template file and does not affect the import verification. During import, it will be determined whether it is an addition or an update based on whether the standard ID is empty. Only standards with edit permissions and in an editable state are supported for updates. The exported update template will filter the selected data standard range. When adding data standards, the system will not automatically create monitoring rules for falling standards. You can select the corresponding monitoring configuration for falling standards when editing the standard.
   STEP 1 Download Configuration Template--Add
   Belonging Data Standard Set, Standard Owner, Effective Time	Support pre-configuring Belonging data standard set, Standard owner, and Effective time information and generating pre-filled content of the template to reduce filling costs. Belonging data standard set: Support selecting the data standard set that the current account has permission to create data standards. If the belonging data standard set is not yet determined, you can also manually fill in the data standard set ID to which each standard belongs in the configuration file. The number of data standard sets that can be selected at one time does not exceed 20. If too many data standard sets are selected, it may exceed the character limit of the Excel drop-down option and cannot display the drop-down option. In this case, you need to enter it yourself. Standard owner: Support selecting users who are normally used under the current tenant. If too many usernames are selected, it may exceed the character limit of the Excel drop-down option and cannot display the drop-down option. In this case, you need to enter it yourself. Effective time: Support setting the effective time of the standard. Permanent: The value of the effective time column is empty (indicating permanent effectiveness). Custom: Effective within the specified time period. Beyond the specified time period, the standard will become invalid. If you need to continue using the standard, you can modify the effective time and republish it.
   Configuration Template	Generate the information columns that need to be configured based on the selected standard template. Click Generate Template to generate a template file in .xlsx format. Note If there is no template, you can click the file name or download the icon to download the .xlsx file; if there is already a template, you can directly execute the upload configuration file and import it. Only batch import is supported under the data standard set with permission to create data standards. Please fill in the data standard set ID with the corresponding permissions to avoid import failure due to permission verification failure.
   STEP 1 Download Configuration Template--Update
   Belonging Data Standard Set, Standard Status	If you need to generate an update type template, you can filter the range of standards to be exported through Belonging data standard set and Standard status. Belonging data standard set: The number of data standard sets that can be selected at one time does not exceed 20. If too many data standard sets are selected, the optional items may be empty after generating the template. Standard status: Support selecting the status of the data standards to be imported. You can choose data standards in In Effect, Yet To Be Effect, Under Revision, Draft, and Expired statuses.
   Configuration Template	Generate based on the selected standard range and standard status. Click Generate Template to generate a template file in .xlsx format. Note If there is no template, you can click the file name or download the icon to download the .xlsx file; if there is already a template, you can directly execute the upload configuration file and import it. Only batch import is supported under the data standard set with permission to create data standards. Please fill in the data standard set ID with the corresponding permissions to avoid import failure due to permission verification failure.
   STEP 2 Upload Configuration File
   Upload File	When filling in the template, please fill it in according to the attribute field definition of the data standard set. During the import process, uniqueness and specification verification will be performed based on the attribute definition. Only a single worksheet .xlsx type file can be selected, supporting up to 10,000 rows, that is, 10,000 standards (excluding the title). The file size does not exceed 20 MB. Title row: Display column names, attribute descriptions, and required attributes, marked with a red *. Standard ID: Cannot be deleted, used to uniquely identify the standard object. It can be empty. If it is empty, it will be treated as a new standard. Belonging data standard set ID: Only batch import is supported under the data standard set with permission to create data standards. Please fill in the data standard set ID with the corresponding permissions (can be queried in the data standard set list) to avoid import failure due to permission verification failure. If the attribute field type is DateTime type, five formats are supported: yyyy/MM/dd, yyyyMMdd, yyyy-MM-dd, hh:mm, and hh:mm:ss. If an attribute is configured as a range value, the filling method is as follows: If Enumeration value input is set, the input content is parsed with an English half-width comma as the separator. The configuration format is: F, M. If Reference lookup table is set, the lookup table code needs to be filled in. The configuration format is: date001. If Interval value is set, the start value and end value are separated by an English half-width comma. If both sides are set, the configuration format is: >=a, <b. If one side is set, the configuration format is: <=b. If Data type is set, the input value must be within the enumeration range. If it has length or precision, the configuration format is: CHAR(20), VARCHAR(20), DECIMAL[38, 18]. If the start effective time and end effective time are empty, the validity period is considered permanent. If a standard attribute references a Dataphin system attribute, please fill in the English name of the object for data blocks and projects, and the object name for users and user groups. Multiple attribute values are separated by a half-width comma. If the filling method of an attribute is a range value of unspecified type, you need to specify the range value type first and then fill in the specific attribute value. If the standard template selected for this import is configured to automatically generate standard codes based on rules, you need to configure the import method of the code. It supports importing according to the configuration file or regenerating based on the coding rules. Please fill in the name of the data classification to facilitate business personnel to understand the relevant information. For example: confidential data. Please fill in the path of the data classification, which can be quickly queried in data security. For example: /personal information/basic personal information/name.
   STEP 3 Import Configuration
   Conflict handling	Verification Field: Standard name (unique within the data standard set, case-insensitive). This field is the unique identification field of the standard. If this field is duplicated, it willbe imported according to the configured duplicate handling policy. Duplicate Handling Policy: The handling policy when the imported standard name duplicates with the existing standard name. Supports two handling methods: Skip if duplicate and Overwrite and update if duplicate. Skip If Duplicate: If the imported standard name duplicates with the existing standard name, the record will be skipped and not processed. Overwrite And Update If Duplicate: If the imported standard name duplicates with the existing standard name, the original record will be deleted after the import is successful. Note Conflict detection is only effective for records with an empty standard ID. If the corresponding standard cannot be found based on the ID, it is considered a verification failure, resulting in an abnormal import. The standard name is unique within the belonging data standard set. If the imported standard name duplicates with an existing standard name, or if there are multiple records with duplicate standard names in a single import, the records will be skipped and not processed according to the handling policy, or the original object will be overwritten with the new record.
   Standard Code Value	When the selected standard template code configuration is set to be automatically generated based on rules, you need to configure the import strategy for the code value. You can choose from two import methods: Import according to the configuration file or Regenerate based on coding rules (ignoring the code value configuration in the import file). Import according to the configuration file: Please fill in the code value. Regenerate based on coding rules: When the template type is Add, during the import process, the code value will be regenerated according to the rules; when the template type is Update, the code value of the updated standard will not be modified during the import process.

4. Click the Start Import button. In the Import Results dialog box, you can perform the following operations:

   Parameter	Description
   Import Results	You can view Import Progress, Executed At, and Number Of Successful Imports.
   Abnormal Records	If there are exceptions such as parsing or verification failures during the import process, the abnormal import records will be displayed, including the corresponding Row Number, Standard Name, Standard Code, and Abnormal Prompt in the original file. On the Abnormal Records tab, you can click Download Abnormal Records to download the corresponding original records as an Excel file.
   Skipped Records	If there are standards with the same name during the import process, the skipped import records will be displayed, including the corresponding Row Number, Standard Name, Standard Code, and Owner in the original file. On the Skipped Records tab, you can click Download Skipped Records to download the corresponding original records as an Excel file.
   Execution Log	You can view and copy the execution log of the import process.

   After completing the import, you can quickly check the results by closing the import configuration dialog box. Closing this box will not stop the import task from executing. To view historical import records, click Batch Operation Records in the standard list. For more information, see Batch operation records.

Batch export standards

The entry point and configured information for batch exporting data standards vary.

1. On the Standard List page, click Batch Import And Export and select Batch Export Standards.

2. In the Batch Export Data Standards dialog box, you can filter the standards you want to export by standard range, status, and template.

   Parameter	Description
   Belonging Data Standard Set	Please select the data standard set to which the data standard belongs. You can choose all standards or specify a range. All standards: Equivalent to selecting all data standard sets that the current account has view permissions for. Specify range: Select the data standard sets to be exported, supporting up to 20 data standard sets.
   Standard Template	Please select the template for exporting standards. Multiple selections are supported, up to 100.
   Standard Status	Please select the status of the data standards to be exported. Multiple selections are supported. You can choose standards in In Effect, Yet To Be Effect, In Publish, Under Revision, Draft, and Expired statuses.

3. Click Confirm to finalize the data standards export. You can then download the generated file from the Batch Operation Records - Batch Export Records section.

Batch operation records

The batch operation records dialog box displays records of batch imports and exports of standards, among other operations.

What to do next

After importing the standards, you can manage the data standards in the standard list. For more information, see Standard list.