The Metadata Center extracts, processes, and centrally stores and manages metadata from various business systems to support data governance and improve data organization, retrieval, and analysis.
Prerequisites
By default, you can collect metadata from relational databases.
To collect metadata with DataWorks, you must purchase a Metadata Collection value-added service.
Before V5.3, you had to initialize the Metadata Center in a metastore tenant to collect data from certain data sources, including AnalyticDB for MySQL 3.0, PolarDB-X (formerly DRDS), SAP HANA, and Hologres. In V5.3 and later versions, you do not need to perform this initialization and can directly configure collection tasks.
To initialize the Metadata Center, the engine of the metastore tenant must be MaxCompute, Hadoop, or Transwarp TDH 9.3.x.
Permissions
Only a super administrator, system administrator, or a global custom role with view permissions for metadata features can view the metadata menus.
Metadata
Metadata is information that describes the characteristics of data. It provides details about the content, origin, format, and structure of data, and is sometimes called "data about data." The purpose of metadata is to make data easier to understand, manage, and use. It helps users identify and locate specific datasets and supports data organization and retrieval. In addition, metadata promotes data consistency and interoperability, and provides the foundation for data governance, compliance, and data quality management. Metadata enables organizations to effectively monitor and analyze their data assets to support better decision-making and gain business insights.
Metadata Classification
Metadata is classified into three types based on its use: technical metadata, business metadata, and management metadata.
Technical metadata stores the technical details of a data warehouse system and supports its development and management. In Dataphin, technical metadata is considered a technical data asset, displaying information such as data domains, subject domains, projects, storage types, storage formats, and lifecycle.
Business metadata describes data in a data warehouse from a business perspective. It provides a semantic layer between users and the underlying systems, enabling business users without deep technical knowledge to understand the data.
Management metadata organizes, integrates, and manages technical and business metadata. It provides metadata services to business and development teams. It also supports the development and maintenance of enterprise business systems and data analytics.
Access the Metadata Center
In the top navigation bar on the Dataphin homepage, choose Governance > Metadata.
On the Metadata page, the left-side navigation pane contains the navigation links for each feature.
First-level menu
Second-level menu
Description
Metadata Collection
Collection Overview
Supports a wide range of data sources for metadata collection, including traditional databases like MySQL and Oracle, and big data storage systems like Hive and Hologres. This page provides an overview of all created collection tasks, data sources, object types, and supported versions, organized by data source type.
Collection Task
A collection task uses a collection adapter to connect to a data source. It collects object metadata, which a built-in parser then processes and stores for display in a unified view.
Collection Instance
A collection instance is a specific run of a collection task. An instance is generated when a task is run manually or triggered by its schedule.
Metadata Management
Metadata Inventory
This page displays the collected metadata in a list. You can query the data from multiple perspectives.
General Settings
Source System
Map the metadata collected from a source to its owner business system. This mapping can then be used for object filtering in the asset inventory and catalog, and for displaying business-system data lineage.
Sampling Settings
Data sampling helps business users better understand the data's structure and aids SQL development.
Profiling and Analysis
Configure the scope of data tables for automatic data profiling. You can also manage resources more effectively by controlling the retention period for data profiling records, the maximum number of concurrent profiling tasks, task execution timeouts, and SET parameters.
Metadata Change Log
Control the scope and number of versions to retain for metadata change records to reduce storage costs.