Dataphin:Manage metadata inventory

Permission description

Super administrators, system administrators, and custom global roles with Metadata Inventory-View permissions can manage the metadata inventory and view metadata details.

Limits

PolarDB-X (formerly DRDS) data source version 1.0 does not support collecting view information.

Manage metadata inventory list

1. In the top menu bar of the Dataphin homepage, select Administration > Metadata.

2. In the left-side navigation pane, select Metadata Management > Metadata Inventory to access the Metadata Inventory page.

3. On the Metadata Inventory page, you can select the Data Table tab or the Dashboard tab to view metadata for different objects. The dashboard tab specifically displays metadata for dashboards within a specified application system. You can switch between application systems in the upper right corner of the page to view data from different systems.

   • Data Table: Allows viewing metadata from the perspectives of all metadata, data sources, and source systems.

     • When viewing all metadata, the display includes metadata name, description, collected data source and encoding, source system, metadata type, latest version, collection task, and update frequency.

     • When a specific data source is specified, the display includes metadata name, database/schema, description, metadata type, and the latest version information. You can click view details next to the list title or view details next to the data source to see the corresponding collection task information, which encompasses data source name, data source encoding, collection task, update frequency, and source system.

     • When a particular source system is selected, the interface shows details such as the metadata name, collected data source, data source encoding, metadata type, latest version, collection task, and update frequency. You can click view details next to the list title or view details adjacent to the source system to access the source system's configuration details, which include the system's name, type, description, operating personnel, and contact information.

   • Dashboard: Viewing is supported only when dashboard metadata has been collected. You can view metadata from the perspectives of all dashboards and workspace/folder.

4. (Optional) The left directory tree offers various filtering methods depending on the object type.

   • Data Table Object: Allows switching between different perspectives to retrieve metadata or searching and filtering target metadata in the upper right corner.

     • Data Source Perspective: View metadata according to the source data source configured for collection. It displays the data source type and the directory tree of the data source. You can expand the data source type and select a directory, and the right list will automatically filter the metadata belonging to the selected directory.

     • Source System Perspective: View metadata according to the source system. Selecting a source system will automatically filter the metadata belonging to it in the right list.

     • Search and Filter: Perform a fuzzy search based on metadata name or description, quickly filter Deleted Data, or filter by source data source, DB/schema, source system, metadata type, or from different perspectives.

       • Data Source Perspective can be filtered by DB/schema and metadata type.

       • Source System Perspective can be filtered by source data source, DB/schema, and metadata type.

     • All Metadata: Click All Metadata to view all metadata.

   • Dashboard Object: Supports retrieving metadata based on all dashboards or workspace perspectives, or searching and filtering target metadata in the upper right corner.

     • Workspace Perspective: View metadata according to the source application system configured for collection. Selecting a workspace will automatically filter out the dashboards belonging to it in the right list.

     • All Dashboards: Click All Dashboards to view all metadata under the specified application system.

     • Search and Filter: Perform a fuzzy search based on the dashboard name or description, or filter based on the workspace/folder (this filter is only available when viewing all dashboards) and source dataset.

5. You can perform various operations on the target metadata object.

   Operation item	Description
   View metadata	This operation is supported only when the asset object is a data table. You can click the name of the collected data source. The system will switch to the data source perspective and locate the data source. If you click the name of the source system to which it belongs, the system will switch to the source system perspective and locate the source system.
   View collection task	This operation is supported only when the asset object is a data table. You can click the name of the collection task to view the collection task configuration information of the object. The dashboard object can quickly jump to view the corresponding collection task by clicking the details icon behind the application system.
   View metadata details	You can click the name of the metadata or the Actions column's View icon to jump to the metadata details page. For more information, see view metadata details.
   View version details	This operation is supported only when the asset object is a data table and the Metadata > Metadata Change Record data source table configuration is enabled. You can click the Actions column's Version Management icon to jump to the version management tab of the metadata details to view information between different versions. For more information, see version management.
   Delete	If the source system data is not deleted, the data will be displayed again after the next collection.

View metadata details

1. Click the name of the target metadata or the view icon in the actions column to access the Metadata Details page.

2. On the Metadata Details page, the displayed metadata information varies based on the asset object.

   Data table

   Shows basic information, table details, field details, partition details, and version management for the metadata.

   • Basic Information: Displays metadata name, data source type, data source and encoding, tenant mode (displayed only for OceanBase), associated schema/database/MaxCompute project, metadata type, and collection task details.

     View Collection Task: Click the collection task name to jump to the collection task list page and automatically filter the corresponding task.

   • Detail: Shows system property information for tables and views (including materialized views), such as business and technical properties.

     • Business Properties: Includes information about the source system to which it belongs.

     • Technical Properties: Includes description, whether it is a lake table, lake table format, storage class, volume, and other details. The collected information varies by data source. For more information, see metadata properties.

       Note

       • For Oracle tables, the storage volume is estimated by the number of table rows and the average space used per row (in bytes). This estimate does not include indexes, unused space, fragmentation, or other factors, and may be lower than the actual occupied space.

       • If the data source table is partitioned, the row count is typically an estimate for reference only. Whether it is updated depends on the metadata update mechanism of each data source system (such as MySQL, OceanBase, etc., which may require manual execution of the analyze command to update).

       • Currently, collecting row count information for the primary table of a partitioned table is not supported.

       • For OceanBase (Oracle tenant) data sources, execute the analyze command to update table row count information. The command is as follows:

         --Replace the table name in angle brackets (<>) with the data table name in the business
         analyze table <schema.tablename>  COMPUTE STATISTICS;

       • For data sources that support creating lake tables, such as Hive, the system determines whether it is a lake table and its corresponding format based on the collected information.

     • DDL Statement: For views or materialized views, you can view the DDL statement from the data source and copy it with a single click.

   • Field Details: For partitioned tables, you can view partition information, which varies for some data sources. For more information, see metadata properties.

     You can search by field name or description, or switch to view field details of different versions.

   • Partition Details: Partition information can only be collected from certain data source types. For more information, see metadata properties.

     Multi-level partitions are displayed based on the combination of each partition level, connected by forward slashes (/).

     Note

     The information on partition record counts and storage volumes is provided for reference purposes only.

   • Version Management: View version records of metadata. You can modify the number of viewable versions in Metadata > Metadata Change Record.

     A new version is generated when changes occur in the source system, based on the collection strategy of the collection task and data changes in the source system. For explanations on generating metadata versions for different data sources, see metadata version.

     For example, if the data update strategy is configured to add new metadata, update changed metadata, and ignore deletions, a new metadata version will not be generated if there are no new or updated data in the source system during two consecutive collections, and only deletions have occurred.

     • You can search for versions by version name.

     • It displays the version number, updates compared to the previous version, table description, and version generation time. You can also view version comparison information.

       • Update Compared to the Previous Version: Compares the metadata changes of the source system in the previous version according to the collection strategy configured in the collection task.

       • Version Comparison: Switch to view historical version records, displaying basic information, changed fields, and all fields. For more information, see the version comparison below.

   Version comparison

   On the Version Comparison page, you can switch between different versions to check for metadata updates.

   • Version Selection: Choose the baseline and comparison versions to view differences between the two.

     • Baseline Version: Select from historical versions generated earlier than the comparison version.

     • Comparison Version: Select from new versions generated later than the baseline version.

   • Basic Information: Displays collection completion time, execution method, DDL statement (viewable for views/materialized views), and table property information affecting version generation. For table property information, see metadata version.

   • Changed Fields: Shows differences between the selected comparison version and the baseline version, including ordinal number, field name, data type, nullability, default value, description, and type of change.

     For instance, if the comparison version modifies a field's description, the change type for that field is an update.

   • All Fields: Displays all field information for both the baseline and comparison versions, including ordinal number, field name, data type, nullability, default value, and description.

   Dashboard

   Presents basic information, property details, and chart information for the dashboard.

   • Basic Information: Shows the dashboard's name, application system type, application system, and collection task details.

     • View Collection Task: Click the collection task name to navigate to the collection task list page and automatically filter the corresponding task.

     • View Metadata: You can quickly search for and switch between different metadata details.

     • View Dashboard: Click View Dashboard to go to the dashboard's detail page in the Quick BI system for more information.

   • Property Information: Displays the dashboard's properties, including business, management, and technical details.

     • Business Properties: Includes the dashboard's description.

     • Management Properties: Shows the BI workspace, folder, owner, last updater, last update time, authorization policy, public status, and public deadline.

       Public Status: Public reports are accessible via links without login, while private reports require login to the Quick BI system.

     • Technical Properties: Lists the source dataset name, total number of charts, and third-party embedding status.

   • Chart Information: Shows the latest collected chart details, including ordinal number, chart name, type, field type, and field name/data type.

     You can search by chart name or filter by chart type, dimension, and measure.