Data standards overview - Dataphin - Alibaba Cloud Documentation Center

Dataphin supports the creation and management of data standards to ensure standardized data production and management, reducing the cost of data application and processing.

Background information

Data standards aim to provide services and support for business, technology, and management. They involve standardized definitions and applications of data and its attributes. Data standards should align with the company's business application and management requirements and reference international, national, regulatory, industry, and internal and external standards.

Business aspect: Standardizing entity data definitions addresses data inconsistency, incompleteness, and inaccuracy, thereby eliminating ambiguity. This ensures a global understanding of data across the enterprise, lowers communication costs among departments and systems, and enhances business process efficiency. With a unified data metric system, business users can readily access and analyze data, fostering data-driven innovation.
Technical aspect: The foundation for enterprise information sharing is built upon unified and standardized data and data structures. Standard data models and data standardization facilitate the development and deployment of new systems, enhancing the efficiency of application system development and deployment. By clearly defining data quality rules, data sources and destinations, and validation rules, data standardization improves overall data quality.
Management aspect: Standardized data definitions clarify data responsibility, ensuring data security and quality. A standardized data metric system supports cross-subject data analysis, improving processing and analysis efficiency. It provides pre-event alerts, mid-event warnings, and post-event reminders for business metrics, enabling data-driven management and quick access to decision-making information.

Prerequisites

Purchase Data Standard value-added services and verify that the current Tenant has the Data Standard module activated.

Scenarios

An IT team in a group has established a data standard for employee gender: use male to represent male, female to represent female, and unknown to represent unknown.

First, the data standard administrator created a gender lookup table and synced it with developers; then created the employee gender standard, referencing the gender lookup table for the range of values, and added a field range validation monitoring rule; finally, they created a mapping rule for the standard.
When creating integration tasks, developers queried the gender lookup table, used the field calculation component to transform non-standard field values such as 0, 1, male, and female in the source table to male, female, and unknown. Then, they referenced the monitoring rules created for the employee gender standard on the quality rules page of the target table and configured the schedule. After the task ran, they discovered through quality alerts that the field value transformation in the source table failed, resulting in non-standard records, which were manually corrected in the target table.

Core concepts

Term	Description	References
Data standard set	A data standard set is a collection of related standards with similar business meanings. It defines management and viewing permissions, approval templates for publishing, and other details for precise control over data standards. Each data standard must belong to a unique set and adhere to the multi-level classification management of its directory.	Create and manage data standard sets
Standard template	A standard template outlines uniform specifications and constraints that data standards should follow, defining standard attributes such as business, technical, and management attributes, which may extend to master data and data quality attributes, among others. To create a data standard, select a standard template and associated data standard set, then populate the standard attributes based on the template details.	Create and manage standard templates
Data standard	Data standards provide services and support for business, technology, and management. The process of data standard management involves the standardized definition and application of data and its attribute information.	Create and manage data standards
Mapping rule	Mapping rules define the association between standard attributes and asset object metadata fields, establishing the basis for standard evaluation on asset objects.	Create and manage mapping rules
Standard evaluation details	Standard evaluation details display the mapping relationship and corresponding evaluation results of data standards and asset objects from both the standard and asset object perspectives, serving as a basis for evaluating asset compliance and promoting development specification rectification.	View standard evaluation details

User roles and permissions

Note

Only standards with "view" permissions can be queried, including: "public" and unpublished standards in the standard set, all standards in the joined standard set, and all standards in the designated visible standard set.

User member type	Role type	Description
Super administrator	System role	The super administrator has the authority to appoint data standard administrators and functions equivalently within the data standard module, with responsibilities that include: Creating, managing, and assigning maintenance personnel, members, visibility, and approval templates for data standard sets. Developing and overseeing standards. Uploading data standards in batches. Adding, managing, and reviewing all lookup tables and their references. Incorporating and overseeing all root words. Managing public attributes of standard sets. Formulating and managing mapping rules. Monitoring details of successful mapping rule executions. Accessing logs of mapping rule executions. Organizing and managing directories of standard sets. Creating and managing customizable public standard attributes. Viewing and administering all valid and invalid mapping relationships between standards and asset objects. Designing and managing standard templates. Establishing and overseeing standard documents.
Data standard administrator	System role	The data standard administrator is granted view, operation, and management permissions for all data standards and is responsible for: Creating and managing data standard sets, assigning maintenance personnel, specifying members, setting visibility, and configuring approval templates. Establishing and overseeing data standards. Uploading data standards in batches. Adding and managing all lookup tables, as well as monitoring their usage. Incorporating and administering root words. Adding and managing attributes for public standard sets. Developing and managing mapping rules. Reviewing detailed logs of successful mapping rule executions. Examining all mapping rule execution logs. Creating and managing directories for standard sets. Formulating and overseeing public standard attributes (customized). Managing all mapped and invalid mapping relationships between standards and asset objects. Designing and managing standard templates. Producing and maintaining standard documents.
Standard set maintenance personnel	Functional role	The responsibilities of standard set maintenance personnel include regular upkeep of standard sets, which encompasses: Editing and deleting standard sets. Managing standard set attributes, including addition, deletion, modification, and retrieval. Creating new standards within the set and managing only those standards they create. Batch uploading of data standards to the respective standard set. Accessing the lookup table list and their references. Reviewing published root words and lookup tables, with the ability to edit and delete those under their purview. Inspecting execution logs and evaluation details of successful mapping rules for accessible standard sets. Viewing public standard attributes and their lists.
Standard set members	Functional role	Members of a standard set primarily focus on establishing standards within the set, either by adding new ones or cloning existing ones. They also manage the standards for which they are responsible. Once created, they have the authority to edit and delete their respective standards. Uploading batches of data standards to the associated standard set. Accessing published root words and lookup tables, with permissions to edit and delete the lookup tables they manage. Reviewing attributes of the system standard set. Examining lookup table references. Observing the public standard attribute lists.
Standard owner	Data role	The standard owner, as one of the four user member types-super administrator, data standard administrator, standard set maintenance personnel, or standard set member-holds additional permissions beyond their role to: Edit, submit for approval, retract, and delete the standards under their purview. Access evaluation details for the standards they manage. Oversee both the established and invalid mapping relationships between their standards and associated asset objects.
Mapping rule owner	Data role	View published mapping rules. Edit and delete mapping rules under their purview. Access execution logs and evaluation details for successful mapping rules within their assigned standard sets. Review lookup table references. Examine public standard attribute lists.
Lookup table owner	Data role	Edit and delete assigned lookup tables. Access lookup table references. Access public standard attribute lists.
Other system roles and ordinary members	System role	Access viewable standard sets to see published standards (effective, pending, expired). Access published root words and lookup tables. Access lookup table references. Access public lists of standard attributes.

Data standard process guide

The data standard process guide helps you become familiar with the configuration path of data standards. The general usage process of data standards is as follows:

Create standard templates->Create and manage data standard sets->Create data standards->Standard publication request->Standard approval->Create mapping rules->View mapping results.

On the Standard page, click Process Guide to access the help guide pop-up. (If no objects are created in the standard set list or data standard list, click View Guide to enter the help guide pop-up)
In the Help Guide pop-up, you can create standard sets, data standards, lookup tables, and root words, as well as view the overall process guide for data standards and configuration case descriptions for mapping rules.

Data standard entry

On the Dataphin home page, select Administration > Standard from the top menu bar.

On the data standard page, the left-side navigation pane displays the entry points for each functional module.

Primary menu	Secondary menu	Description
Data standards	Data standards	Data standards list, supports viewing all standards in the joined standard set.
Data standards	Standard sets	Standard sets list.
Implementation evaluation	Mapping rules	List of mapping rules configured based on standard set attributes and asset object metadata attributes, including the configuration of the effective range of mapping rules.
	Mapping relationships	List of mapping relationships between asset objects and data standards, including confirmed mapping relationships and relationships designated as invalid mappings. Supports two modes: summary view by asset granularity or full tile view of mapping relationships. It also supports configuring quality monitoring rules for mapped relationships.
	Standard evaluation details	Displays the evaluation details of all standards and asset objects involved in mapping and evaluation globally, supporting views from both the standard perspective and the asset object perspective.
	Mapping execution records	Displays the execution details and logs of mapping rules.
Reference data	Lookup tables	A list of available lookup tables, complete with details for review.
	Root Words	A comprehensive list of root words, including details that can be viewed for each.
	Standard Documents	A list of standard documents, detailing the available documents for review.
General configuration	Public standard attributes	List of public standard attributes, including customized public attributes and system public attributes.
	Standard templates	List of standard templates, with details of each standard template's attributes and the data standards created using the template available for viewing.
	Identification features	List of identification features, including customized identification features and system-built identification features.