Dataphin:Manage metadata inventory

Permission description

Super administrators, system administrators, and custom global roles with Metadata Inventory - View permissions are able to manage the metadata inventory and view metadata details.

Limits

PolarDB-X (formerly DRDS) data source version 1.0 does not support the collection of view information.

Manage metadata inventory list

1. On the Dataphin home page, navigate to the top menu bar and select Administration > Metadata.

2. In the left-side navigation pane, select Metadata Management > Metadata Inventory to access the Metadata Inventory page.

3. On the Metadata Inventory page, you can select either the Data Table tab or the Dashboard tab to access metadata information for various objects. The Dashboard tab, in particular, presents metadata associated with the dashboard of the selected application system. You can switch between different systems using the application system dropdown located at the top-right corner of the page.

   • Data Table: View metadata from the perspectives of all metadata, data sources, and source systems.

     • Under all metadata, it displays metadata name, description, collected data source and encoding, source system, metadata type, latest version, collection task, and update frequency.

     • When a specific data source is specified, the display includes metadata name, database/schema, description, metadata type, and the latest version information. You can click view details beside the list title or view details next to the data source to see the corresponding collection task information, which covers data source name, encoding, collection tasks, update frequency, and the source system.

     • When you specify a particular source system, the interface shows details such as the metadata name, collected data source, data source encoding, metadata type, latest version, collection task, and update frequency. You can click view details next to the list title or view details adjacent to the source system to see the source system's configuration details, which include the system name, type, description, operating personnel, and system contact.

   • Dashboard: Viewing is available only when dashboard metadata is collected. You can view metadata from the perspectives of all dashboards and workspace/folder.

4. (Optional) The left directory tree offers various filtering methods based on different objects.

   • Data Table Object: Switch between different perspectives to retrieve metadata or search and filter target metadata in the upper right corner.

     • Data Source Perspective: View metadata according to the source data source's collected configuration. This perspective shows the data source type and its directory tree. Expanding a data source type and selecting a directory will automatically filter the metadata list on the right to show only those belonging to the current directory.

     • Source System Perspective: View metadata according to the source system of the data. Selecting a source system will automatically filter the metadata list on the right to show only those belonging to the chosen source system.

     • Search and filter: You can conduct a fuzzy search using the metadata name or description, swiftly filter out Deleted Data, or apply filters by data source, DB/schema, source system, metadata type, or various other perspectives.

       • Data Source Perspective can be filtered by DB/schema and metadata type.

       • Source System Perspective can be filtered by source data source, DB/schema, and metadata type.

     • All metadata: To view all metadata, click All Metadata.

   • Dashboard Object: Retrieve metadata based on all dashboards or workspace perspectives, or search and filter target metadata in the upper right corner.

     • Workspace Perspective: View metadata according to the source application system's collected configuration. Selecting a workspace will automatically filter the list on the right to show only the dashboards belonging to the chosen workspace.

     • All Dashboards: You can click All Dashboards to view all metadata associated with the selected application system.

     • Search and Filter: Perform a fuzzy search based on the dashboard name or description, or filter by workspace/folder (this filter is only available when viewing all dashboards) and source dataset.

5. You can perform various operations on the target metadata object.

   Operation item	Description
   View metadata	Supported only when the asset object is a data table. You can click the name of the collected data source, and the system will switch to the data source perspective and locate the data source. If you click the name of the source system to which it belongs, the system will switch to the source system perspective and locate the source system.
   View collection task	Supported only when the asset object is a data table. You can click the name of the collection task to view the collection task configuration information of the object. For the dashboard object, you can click the details icon behind the application system to quickly jump to view the corresponding collection task.
   View metadata details	You can click the name of the metadata or the Actions column's View icon to jump to the metadata details page. For more information, see view metadata details.
   View version details	Supported only when the asset object is a data table. You can click the Actions column's Version Management icon to jump to the version management tab of the metadata details. For more information, see version management.
   Delete	If the source system data is not deleted, the data will be displayed again after the next collection.

View metadata details

1. Click on the target metadata name or the view icon in the actions column to access the Metadata Details page.

2. On the Metadata Details page, the displayed metadata information varies based on the asset object.

   Data table

   Shows basic information, table details, field details, partition details, and version management for the metadata.

   • Basic Information: Displays metadata name, data source type, data source and encoding, tenant mode (displayed only for OceanBase), associated schema/database/MaxCompute project, metadata type, and collection task details.

     View Collection Task: Click the collection task name to jump to the list page and automatically filter to the corresponding task.

   • Detail: Shows information on system properties for tables and views, encompassing both business and technical properties, including materialized views.

     • Business Properties: Includes information about the source system to which it belongs.

     • Technical properties: This includes descriptions, storage class, volume, and additional details. The range of information gathered varies with the data source. For more information, see metadata properties.

       Note

       • For Oracle tables, storage volume is estimated based on row count and average space per row, excluding index statistics, unused space, fragmentation, etc. The estimate may be lower than the actual space occupied.

       • If the data source table is partitioned, the row count is for reference only, as collecting row count information for the primary table of a partitioned table is not supported.

       • When the data source is OceanBase (Oracle tenant), you must execute the analyze command to collect information on the table row count. The command is as follows:

         --Replace the table name in angle brackets (<>) with the data table name in the business
         analyze table <schema.tablename>  COMPUTE STATISTICS;

     • DDL Statement: For metadata types such as views or materialized views, you can view and copy the DDL statement from the data source for the current view or materialized view with a single click.

   • Field Details: For partitioned tables, you can view the partition information, which varies depending on the data source. For more information, see metadata properties.

     Search by field name or description and switch to view field details for different versions.

   • Partition Details: Partition information can only be collected from certain types of data sources. For more information, see metadata properties.

     Multi-level partitions are displayed based on the combination of each partition level, separated by forward slashes (/).

     Note

     Please note that the partition record count and storage volume information serve only as references.

   • Version Management displays the latest 50 version records. A new version is created whenever there are changes to the metadata in the source system. This is determined by comparing the most recent source system against the previous one, based on the collection task's policy configuration and any changes in the source system data. For explanations on how metadata versions are generated for different data sources, see metadata version.

     For instance, if the data update policy is set to add new metadata, update changed metadata, and ignore deletions, then a new metadata version will not be generated if only deletions occur between collections.

     • Search for versions by version name.

     • View version number, updates compared to the previous version, table description, and version creation time. You can also see version comparison details.

       • Updates Compared to the Previous Version: Compare metadata changes in the source system of the previous version based on the collection task's policy.

       • Version Comparison: Switch to view historical version records, showing basic information, changed fields, and all fields. For more details, see the version comparison below.

   Version comparison

   You can toggle between different versions on the Version Comparison page to verify updates to the metadata.

   • Version Selection: Choose the baseline and comparison versions to see differences in metadata between them.

     • Baseline Version: Select from historical versions created earlier than the comparison version.

     • Comparison Version: Select from newer versions created after the baseline version.

   • Basic information: Shows the time of collection completion, execution method, DDL statement (supports viewing for views/materialized views), and table property details that influence version creation. For table property information, see metadata version.

   • Changed Fields: Displays differences between the comparison and baseline versions, including ordinal number of changed fields, field name, data type, nullability, default value, description, and change type.

     For example, if the comparison version modifies a field description, the field's change type is updated.

   • All Fields: Shows all field information for both the baseline and comparison versions, including ordinal number, field name, data type, nullability, default value, and description.

   Dashboard

   Displays basic information, property details, and chart data for the dashboard.

   • Basic Information: Shows the dashboard name, application system type, application system, and collection task details.

     • View Collection Task: Click the collection task name to navigate to its list page and filter to the relevant task.

     • View Metadata: You can easily search for and switch between different metadata details.

     • View Dashboard: To see detailed information, click View Dashboard to access the specific dashboard page within the Quick BI system.

   • Property Information: Provides property details of the dashboard, including business, management, and technical aspects.

     • Business Properties: Includes the dashboard's description.

     • Management Properties: Details the BI workspace, folder, owner, last updater, last update time, collaborative authorization policy, public access status, and public deadline.

       Public Access: Public reports are accessible via a link without login, while private reports require login to the Quick BI system.

     • Technical Properties: Lists the source dataset name, total number of charts, and third-party embed status.

   • Chart Information: Shows the most recently collected chart data, including the chart's ordinal number, name, type, field type, and field name/data type.

     Search by chart name or filter by chart type, dimension, and measure.