Create and manage data standards - Dataphin - Alibaba Cloud Documentation Center

Data standards are used to support business, technology, and management requirements. Managing data standards involves defining and applying standards to data and its attributes. This topic describes how to create and manage data standards.

Permissions

Super administrators and data standard administrators can create or clone data standards in any data standard set.
Data standard set maintainers and members can create or clone data standards in the data standard sets to which they belong.
Super administrators, data standard administrators, and data standard owners can manage the data standards for which they are responsible. These operations include editing, publishing, deleting, unpublishing, and editing associated standards.

Limits

You can create a maximum of 1,000 data standards in a single data standard set.

Standard list

On the Data Standard page, you can focus on specific data standard sets that you follow or are associated with. This lets you quickly view an overview of the standards within those sets. The data standard list provides two display modes: edit and view. On the Standard List page, you can click the icon in the upper-right corner to switch between modes.

In view mode, you can view configured data standard set folders and data standard information, but you cannot perform operations. This mode is suitable for browsing.

In edit mode, you can view configured data standard set folders and data standards. You can also perform operations such as creating, editing, deleting, cloning, publishing, and unpublishing standards, and editing and deleting data standard set folders. Only users with data standard management permissions for the current data standard set can enter edit mode. The following figure shows an example of edit mode.

Area	Description
① Filter and search area	You can search for a data standard by its name, encoding, or ID. You can also filter by standards you own, the owner, the data standard set, the standard type, or the standard template.
② Data standard set	The data standard set folder classifies and manages data standard sets and data standards from a business or organizational perspective. Click the expand all or collapse all icon to expand or collapse the data standard sets. You can search by the name of the data standard set folder, or the name or encoding of the data standard set. You can also filter by Joined By Me or Maintained By Me. All Data Standards: Click All Data Standards to view a complete list of standards for which you have view permissions. This helps you get a full overview of all data standards more quickly. Data Standard Set: Click a data standard set in the selected data standard set folder. The list of standards belonging to that set is displayed on the right. Data Standard Set Folder: Displays all data standard sets that belong directly to this folder level and its subdirectories. To create a folder, see Manage data standard set folders. Hover your mouse over a data standard set folder or a data standard set to view its information. For a folder, this includes the folder name, folder encoding, total number of data standard sets, the last updater, and the last update time. For a data standard set, this includes the set name, set encoding, the last updater, and the last update time.
③ Standard list	The information displayed and the operations supported for a data standard vary based on its status. For more information about the operations, see Supported operations for data standards. When the status of a standard is In Effect, Pending, or Expired, the list displays the standard's name, its folder/data standard set (shown when viewing All Data Standards), encoding/ID, type, description, version, effective period, owner, last updater, and last update time. When the status of a standard is Publishing, the list displays the standard's name, its folder/data standard set (shown when viewing All Data Standards), encoding/ID, type, description, publish type, publish status, effective period, owner, last updater, and last update time. When the status of a standard is Under Revision or Draft, the list displays the standard's name, its folder/data standard set (shown when viewing All Data Standards), encoding/ID, type, description, effective period, owner, last updater, and last update time.

Supported operations for data standards

Status	Operation
In Effect, Pending, Expired	Supports View, Edit, Clone, Edit Associated Standards, Edit Associated Documents, Subscribe to Standard Changes, and Unpublish. You can also click the Subscribe To Standard Changes or Unpublish button at the bottom to batch unpublish or subscribe to changes for standards that are in effect, pending, or expired.
Publishing	Supports View, Go to Approval Task, Republish, Revoke Publishing, Clone, Edit Associated Standards, Edit Associated Documents, and Subscribe to Standard Changes.
Under Revision, Draft	Lets you view, edit, submit for publishing, clone, edit associated standards, edit associated documents, subscribe to standard changes, and delete. Alternatively, click the Subscribe To Standard Changes, Publish, or Delete button at the bottom to batch publish, delete, or subscribe to changes for standards that are being revised or are in a draft state.

Operation	Description
View	Click the name of a standard or the View icon in the Actions column to view the standard's Basic Information, Monitoring Rules, and History. You can switch between historical versions on the Basic Information and Monitoring Details pages. Basic Information: Includes attribute information, business information, intelligent mapping settings, and associated information. Attribute Information: Displays the attribute configuration of the data standard. Business Information: Displays the data standard set and folder, effective period, owner, and description. Intelligent Mapping Settings: Displays information about detection features. Associated Information: Associated information is not versioned. You can view it only when the latest version is selected. It includes associated standards, associated lookup tables, and associated documents. Associated Standards: View information about associated data standards, including the standard name, standard encoding, folder and data standard set, owner, and association relationship. You can also configure associated data standards. For more information, see Edit associated standards. Associated Lookup Tables: Information about associated lookup tables is automatically generated based on configured standard attribute values. It includes the lookup table name, lookup table encoding, associated standard attribute, owner, and association relationship. Associated Documents: View information about associated standard documents, including the document name, description, folder, and creator. You can also configure associated standard documents. For more information, see Edit associated documents. Monitoring Rules: Displays the rule name, monitoring type, add method, rule type, validation rule, and case sensitivity. You can click the View icon in the Actions column to view the details of the monitoring rule. Version History: Displays the version number, status, last modified by, creation time, publish time, and release notes for each version. You can also perform the View Version Details and Version Comparison operations on this page. View Version Details: Click to switch to the Basic Information page for the corresponding version. Version Comparison: You can only compare published production version numbers. You can switch to view historical version records. Different colors in the configuration information represent different operations. Red indicates deleted information. Green indicates new information. Yellow indicates that the parameter is unchanged, but its value has been modified. A mixed color indicates that the corresponding value cannot be retrieved. Note In the production environment, standards that are in effect, pending, or expired display their published historical version information.
Edit	You cannot edit a standard that is in the Publishing state. After a standard is created, you cannot edit its encoding. Editing a standard that is In Effect, Pending, or Expired creates a new version of the standard with the status Under Revision. The revised standard does not affect the published version until it is published again.
Submit For Publishing	If release approval is enabled for the data standard set, the submission must be approved. After approval, the standard's status changes to In Effect or Pending. If approval is not enabled, the standard is published directly to the production environment after passing validation. Note If a standard has a pending unpublish approval request, you must revoke the unpublish request before you can submit it for publishing. When you batch submit standards for publishing, the operation applies only to standards for which you have permissions. Other standards are skipped. If the data standard set is configured to merge batch operations into a single approval task, one approval task is created for all selected standards. This task can be approved or rejected as a whole. If merging is not enabled, a separate approval task is created for each selected data standard. Each standard's approval can be handled independently. After a standard is successfully submitted and published, you can create mapping associations for the standard.
Clone	Quickly clone a standard.
Delete	You can delete standards that are in Draft or Under Revision state. This operation cannot be undone. Proceed with caution. Note When you batch delete standards, the operation applies only to standards for which you have permissions. Other standards are skipped. You can view the information in the batch operation records. For more information, see Other operation records.
Go To Approval Task	Go to the approval task for the standard.
Republish	If publish/unpublish approval is enabled for the standard's data standard set and the previous approval was rejected, you must go through the approval process again to republish. If the configuration of the data standard set has changed, you cannot republish. You must revoke the current publishing request, modify the standard's configuration, and then initiate a new approval. This applies to standards whose data standard set does not have publish/unpublish approval enabled, or where approval was enabled and granted, but the publishing process failed. Note If merge approval is enabled for the data standard set to which the selected standard belongs, and a previous approval flow for the standard was rejected or revoked, this republish action triggers a new approval flow that includes all standards from the previous flow. We recommend that you first View Historical Approval Tasks to confirm that the scope of the standards to be submitted is as expected. If you do not want the associated standards to be republished, we recommend that you first revoke the publication and then submit a new publication request.
Revoke Publishing	If the task is Pending Approval, the approval task is automatically revoked, and the object being published is reverted to its state before submission. If the task was Rejected, the object being published is reverted to its state before submission. Note If the data standard set that contains the current standard has the merge approval tasks feature enabled, the entire approval task will also be revoked. You can click View Approval Task to confirm whether this is the expected outcome before you revoke the publication.
Edit Associated Standards	Configure associated data standards to easily view related standard relationships. For example, a standard for a business metric can be associated with the standards for its statistical dimensions. Associated standards are not versioned. You can view the association relationship only when the latest version is selected. Edit: Click the Edit button to add or delete associated standards. Click the Save button to complete the changes.
Edit Associated Documents	Configure associated standard documents to easily view related document relationships. Associated documents are not versioned. You can view the association relationship only when the latest version is selected. Edit: Click the Edit button to add, download, or delete associated documents. Click the Save button to complete the changes.
Subscribe to Standard Changes	Subscribing to standard status changes lets you promptly sync with any updates and make business adjustments as needed. You can view your standard subscription information in My Subscribed Assets. For more information, see View my subscribed assets. Subscribe to a single standard's changes: Select the standard you want to subscribe to. In the Add Standard Change Subscription dialog box, configure the standard status changes and notification method. For more information, see Add a subscription. Batch subscribe to standard changes: Select the standards you want to subscribe to. In the Batch Subscribe To Standard Changes dialog box, configure the conflict handling policy for already subscribed standards, the standard status changes, and the notification method. For more information, see Add a subscription. Note Select at least one standard status change and one notification method. When you batch subscribe to standard changes, the operation applies only to standards for which you have view permissions. Otherwise, the subscription will fail. Make sure you have view permissions for all standards in the data standard set.
Unpublish	If unpublish approval is enabled for the data standard set, the unpublish request must be approved. After approval, the standard's status changes to Draft. If approval is not enabled, a draft is generated directly after submission and validation. Note If a standard has a pending publish approval request, you must revoke the publish request before you can unpublish it. When you batch unpublish standards, the operation applies only to standards for which you have permissions. Other standards are skipped. If the data standard set is configured to merge batch operations into a single approval task, one approval task is created for all selected standards. This task can be approved or rejected as a whole. If merging is not enabled, a separate approval task is created for each selected data standard. Each standard's approval can be handled independently. After a standard is successfully unpublished, all its mapping relationships are automatically deleted. Quality rules created by referencing the standard's monitoring configuration will fail to run. You can modify the standard and submit it for publishing again.

Create a data standard

A data standard must be created in a specific data standard set based on a specific standard template.
The data standard set determines the classification, permission controls, and release approval flow of the standard. The standard template determines the attributes and configurable monitoring rules that are required to create the data standard.
You can create a data standard in one of the following three ways:
- Method 1: On the global standard list page, you can click the Create Standard button in the upper-right corner. Select the data standard set and standard template for the new standard to open the standard details configuration page.
- Method 2: On the data standard list page for a single data standard set, if the set has a default standard template configured, you are directed to the standard details configuration page that is generated from the attributes of the default template. If the data standard set does not have a default standard template, you must select a standard template before you can open the create standard page.
- Method 3: On the standard template list page, you can click the Create icon for a standard template. The current template is selected by default. After you select a data standard set, you are directed to the create standard page.

Procedure

On the Dataphin home page, click Administration > Standard in the top menu bar.
The following steps describe how to create a standard from the global standard list page. In the navigation pane on the left, click Standard, and then click + Create Standard.
In the Create Data Standard dialog box, select a data standard set and a standard template.
- Standard Set: Select the data standard set to which the data standard belongs.
- Standard Template: If the selected data standard set has a default standard template, it is selected by default. If no default template is set, you must select a standard template. For more information, see Create and manage standard templates.

Click Next to open the create standard page and configure the parameters.

To switch the standard template, you can click Switch in the upper-right corner to select a different one.

Important

If you switch the template after you have configured information for the data standard, all existing information is cleared. This action cannot be undone. Ensure that your configuration is complete and meets the attribute value specifications.

Parameter	Description
Attribute Settings
Attribute Information	The rules for filling in attribute values depend on the attribute settings of the standard template. For example, if the Skip Execution Date attribute has a field value type of Enumerated Value (Single-select) and references the Chinese Legal Holidays lookup table, then when you create the data standard, this attribute will be a drop-down list. The options will be the values from the Chinese Legal Holidays lookup table. Standard Encoding Value: If the standard encoding for the current template is automatically generated based on an encoding rule, the system displays the generated encoding value. You can click Regenerate to generate a new value, or click Switch to Custom Edit to enter an encoding value that follows the template's encoding rule constraints. System Attribute Settings: Range: If the attribute's field value type is Range, configure it as follows: Enter enumerated values: Use this when the value range is enumerable. Separate multiple values with commas (,). The maximum length is 5,000 characters. Interval value: Use this when the value range is a continuous number or date. You can select from >, >=, <, <=, and No Limit. If you select No Limit for both the maximum and minimum values, no configuration is needed. Reference lookup table: Use this when the attribute value range is dynamic or needs to follow a specific constraint. You can only select lookup tables that are in the Published state. For more information about attribute settings, see Custom attributes. Built-in Attributes: If you reference Dataphin's built-in system attributes, the constraints are as follows: If the attribute configuration references System Attribute - Data Type, select the data type used to store the field in the database. If you select CHAR(n) or VARCHAR(n), configure the length. The default for CHAR is 255, and you can select an integer from 1 to 255. The default for VARCHAR is 1,000, and you can select an integer from 1 to 65,535. If you select DECIMAL(p,s), configure the precision. The default for DECIMAL is [38,18]. You can select an integer from 1 to 38 for precision, and an integer from 0 to 18 for scale. The scale must be less than the precision, or the input is invalid. If the attribute configuration references System Attribute - Value Range, configure the value constraint range based on the attribute's field value type and range type. If the attribute configuration references System Attribute - Is Nullable/Is Empty String/Is Unique Value, you can set the constraint to Yes or No. If the attribute configuration references System Attribute - Data Classification/Data Sensitivity Level, select the required data classification category and data sensitivity level to better manage data. For information on how to create data classifications and data sensitivity levels, see Manage data classifications and Manage data sensitivity levels. Note If a built-in system attribute is referenced by a data standard, its associated mapping monitoring is also referenced. If you have not activated the corresponding module, you cannot reference the mapping monitoring configuration of the system attribute. You can only reference the attribute configuration. The attributes correspond to different functional modules as follows: Value Range, Is Nullable, Is Empty String, and Is Unique Value have built-in content quality monitoring. You must activate the Asset Quality module for the monitoring to take effect. Data Classification and Data Sensitivity Level require the Asset Security module to be activated for the monitoring to take effect. When referencing a public system attribute, you can choose whether to automatically add the corresponding mapping monitoring configuration. The system generates validation details based on the attribute value and meaning. You can manually modify the configuration as needed. If you do not need mapping audits, you can manually deselect the monitoring configuration. Note: The selected state of the monitoring configuration next to the attribute corresponds to the rules in the mapping monitoring configuration list below.
Business Information
Data Standard Set And Folder	The default data standard set name.
Effective Period	Set the effective period for the standard. Permanent: The standard is permanently effective. Custom: The standard is effective for a specified period. After the period ends, the standard's status changes to Expired. To continue using the standard, you can modify the effective period and republish it.
Owner	Select the owner of the standard. The owner must be a maintainer or member of the data standard set and can edit and delete the standard.
Description	Enter a custom description. The maximum length is 256 characters.
Bid Loss Monitoring Configuration Monitoring rules include metadata monitoring and content quality monitoring inherited from referenced system attributes, along with custom-added quality monitoring rules. For more information about metadata and content quality monitoring, see Metadata monitoring and Content quality monitoring. To create a quality monitoring rule, see Quality monitoring rule configuration. You can click the button to view a description of the monitoring configuration.
Quality monitoring rule configuration	If you have purchased the Asset Quality module, you can add custom quality monitoring rules, such as Field Uniqueness Validation. After configuration, all asset objects mapped to the current standard can quickly reference the monitoring rules configured here from the quality rule configuration page. This helps enforce data standard constraints and enables batch creation of quality monitoring rules. You can configure a maximum of 100 monitoring rules for each data standard. You can view, edit, and delete monitoring rules. When creating a quality rule for a Dataphin data table, you can reference the quality monitoring rules configured here to quickly generate quality rules in batches. On the standard mapping relationship page, for each valid mapping relationship, you can quickly reference the monitoring rules configured here for the standard in the relationship and generate the corresponding quality monitoring rules. Note If a quality monitoring rule is deleted from the standard, the quality rule configurations that reference it will become incomplete. Proceed with caution based on your business needs. If a quality monitoring rule in the standard is modified, the quality rules that reference it are updated synchronously. However, custom quality configuration content is not modified. Referencing a public attribute with monitoring automatically adds the corresponding monitoring rule, such as metadata monitoring. The editable information for quality rules generated in different features varies. For more information, see Quality rule editing instructions.
Intelligent Mapping Settings
Feature	After you configure detection features associated with a standard, the system provides intelligent recommendations for mapping relationships between standards and asset objects when mapping rules are executed. This is based on the defined feature expressions, data content, and metadata such as object descriptions, which reduces manual configuration effort. You can select a maximum of 20 detection features. The relationship between multiple detection features is OR. Click the View icon next to a detection feature to view its details in the View Feature Details dialog box.
Associated Information
Associated Standards	View information about associated data standards, including the standard name, standard encoding, folder and data standard set, owner, and association relationship. Add Associated Standard: Click the Add button. In the Add Associated Standard dialog box, associate data standards. You can associate a maximum of 50 standards. Click OK to add the associated standards to the list. The association is saved only after you click the Save or Save And Publish button at the bottom. View Standard: Click the name of a standard to view its basic information, monitoring rules, and history in the View Standard panel. Delete Associated Standard: Click the Delete icon in the Actions column to remove the associated standard.
Associated Lookup Tables	View information about associated lookup tables, including the lookup table name, lookup table encoding, owner, and association relationship. View Lookup Table: Click the name of a lookup table or the View icon in the Actions column to view the lookup table details and reference information. Edit Associated Lookup Table: Associated lookup tables are automatically generated based on configured standard attribute values. You can modify the associated lookup tables by changing the attribute values.
Associated Documents	View information about associated standard documents, including the document name, description, folder, and creator. Add Associated Document: Click the Add button. In the Add Associated Document dialog box, associate documents. You can associate a maximum of 50 documents. Click OK to add the associated documents to the list. The association is saved only after you click the Save or Save And Publish button at the bottom. Preview Document: You can preview PDF documents online. Download Document: You can download standard documents to your local machine to view them. Delete Associated Document: Click the Delete icon in the Actions column to remove the associated document.

Click Save. The new standard is then displayed in the standard list.
Click Save And Publish. In the Submit Standard For Publishing dialog box, enter a description in the Approval Remarks field and click OK to publish the standard to the production environment.

Quality monitoring rule configuration

Parameter		Description
Basic Information	Rule Name	Enter a custom rule name based on your business needs. The maximum length is 128 characters.
	Description	Enter a custom description for the quality rule. The maximum length is 128 characters.
	Template Source	System Template: The built-in parameters of the template are configurable. This is suitable for creating general-purpose rules. Custom Template: The template has preset parameters that do not need to be configured. This is generally used for creating rules that contain business logic.
	Rule Template	Rule templates include Completeness, Uniqueness, Validity, and Stability. Completeness: Includes Null Value Validation and Empty String Validation. Uniqueness: Includes Uniqueness Validation, Field Group Count Validation, and Duplicate Value Count Validation. Validity: Includes Column Format Validation, Column Length Validation, Column Value Domain Validation, and Reference Table Validation. Stability: Includes Column Stability Validation and Column Volatility Validation. For more information about the templates, see Template type descriptions.
Rule Configuration		Configure the details of the monitoring rule. To configure a template for a validation field, it is automatically selected based on the field or metric mapped to the standard and does not need to be configured here. For other settings, see Data table parameter settings. Note: You can only modify the rule configuration in the data standard. After a quality rule references this configuration, you cannot modify the rule configuration from the quality page. However, you can modify the rule name, rule strength, scheduling configuration, and effective status, and you can run and test the rule.
Validation Configuration		The validation configuration varies depending on the selected template. For more information, see Validation configuration description. Note: You can only modify the validation configuration in the data standard. After a quality rule references this configuration, you cannot modify the validation configuration from the quality page.