All Products
Search

This Product

    Dataphin:Metadata Collection

    Document Center

    Dataphin:Metadata Collection

    Last Updated:Aug 28, 2026

    Dataphin supports metadata collection from a wide range of data sources, including traditional databases such as MySQL and Oracle, big data storage such as Hive and Hologres, and application systems. You can see at a glance the number of collection tasks and the object types collected for each data source and application system.

    Prerequisites

    • To use an application system as a data source, you must first create the application system in Management Center > Data Source Management > Application System.

    • Dataphin supports metadata collection for relational databases by default. To collect metadata from other data source types, you must purchase the corresponding features.

    • To collect metadata from a DataWorks application system, you must purchase the Metadata Collection value-added service.

    • Before V5.3, some data sources, including AnalyticDB for MySQL 3.0, PolarDB-X (formerly DRDS), SAP HANA, and Hologres, required an initial configuration of the Metadata Center in the metadata warehouse tenant to enable metadata collection. In V5.3 and later versions, this initial configuration is no longer necessary, and you can configure collection tasks directly.

    Metadata Collection Workflow

    If your data source's network is isolated from the Dataphin cluster's network, metadata collection relies on the registered scheduling cluster feature. Dataphin writes the collected data to an object storage system it uses, such as OSS, which acts as a staging area before Dataphin ingests the data. This workflow incurs additional storage costs.

    Procedure

    1. In the top menu bar of the Dataphin homepage, choose Governance > Metadata.

    2. In the left navigation pane, choose Metadata Collection > Collection Overview.

    3. On the Welcome to metadata collection and management page, Dataphin uses cards to display information for each data source and application system, including the number of configured collection tasks and the supported object types.

      • Data source: Dataphin supports various data source types, such as relational databases and big data storage. For a complete list, see Data sources supported by Dataphin.

        Supported versions for MySQL, Oracle, and Hive (with MySQL metastore and HMS metadata) include:

        • MySQL: MySQL 5.1.43, MySQL 5.6/5.7, MySQL 8, and RDS MySQL.

        • Oracle: Oracle 11g, 12c, 18c, 19c, 21c, and 23c.

        • Hive (MySQL metastore and HMS metadata): CDH 5.x Hive 1.1.0, EMR 3.x Hive 2.3.5, EMR 5.x 3.1.x, CDH 6.x Hive 2.1.1, FusionInsight 8.x Hive 3.1.0, CDP 7.x Hive 3.1.3, and AsiaInfo DP 5.x Hive 3.1.0.

      • Application system: Supports Quick BI and DataWorks.

    4. You can create collection tasks for a target data source or application system directly from this page.

      Create collection task: Hover over a card to quickly create a collection task. For more information, see Create and manage metadata collection tasks.

      Note

      You can configure only one collection task per data source. However, you can configure separate collection tasks for the development and production environments of the same data source.