Data lineage tracks how data flows through your pipelines — which tables feed into which, and how fields relate across them. Use it to trace data origins and assess the impact on upstream and downstream jobs when a job fails.
Hologres integrates with DataWorks Data Map to provide data lineage management. Data Map is an enterprise data catalog management module built on metadata. It provides features such as viewing metadata details, data lineage, and data category management. Data Map helps you find, understand, and use data more effectively. For more information, see Data Map overview.
Limitations
Only Hologres V1.1 or later supports Data Map. If your instance runs an earlier version, upgrade it. For assistance, see Common upgrade preparation failure errors or join the Hologres DingTalk group to provide feedback. For more information, see How do I get more online support?.
Only DataWorks Standard Edition or higher supports viewing lineage information.
Lineage information appears in Data Map one hour after you configure the Hologres metadata collector.
View Hologres data lineage
Prerequisites
Before you begin, ensure that you have:
A Hologres instance running V1.1 or later
DataWorks Standard Edition or higher
Note: Lineage information appears in Data Map one hour after you configure the Hologres metadata collector.
Step 1: Ingest Hologres metadata
Use the metadata acquisition feature to import metadata from your Hologres data source into Data Map.
After acquisition completes, the Data Overview page shows statistics and table information for all Hologres databases that have collectors configured in the current region. For more information, see Data Overview and Metadata acquisition.
To find a specific table after ingestion, see Metadata retrieval.
Step 2: View lineage details
Click the target table name to open its details page.
On the details page, review basic information and output information, then switch to the Lineage tab. The following image shows an example for a Hologres internal table.
For more information, see View table details.
View lineage with MaxCompute
Data Map also shows lineage between MaxCompute and Hologres tables. Click a Hologres foreign table link to navigate to the details page of the corresponding MaxCompute table.
To navigate from a foreign table link to the MaxCompute table details page, the MaxCompute project must be attached to your target DataWorks workspace.
The following image shows an example where data is written from a Hologres foreign table to a MaxCompute table via the DataWorks scheduling module. Switch to the Field Lineage tab to view field-level lineage.
View lineage with Flink
View lineage between Hologres and Flink in the Flink console. For more information, see View lineage relationships.