Data lineage shows the relationships between tables and fields. You can use data lineage to trace data sources and manage data. If a job is abnormal, data lineage also helps you analyze the impact on upstream and downstream jobs. Because Hologres is deeply integrated with DataWorks, you can manage data lineage for Hologres in the Data Map module of DataWorks.
Background information
DataWorks Data Map is a module for managing enterprise data catalogs based on metadata. It provides features such as viewing metadata details, data lineage, and data category management to help you find, understand, and use data. For more information, see Data Map overview.
Limits
Only Hologres V1.1 and later versions support the Data Map feature. If your instance is an earlier version, see Common errors when preparing for an upgrade or join the Hologres DingTalk group for feedback. For more information, see How do I get more online support?.
Lineage information is available only in DataWorks Standard Edition and later versions.
Lineage information becomes available approximately one hour after you configure the Hologres metadata collector in Data Map.
View Hologres data lineage
Follow these steps to view the lineage between Hologres tables in DataWorks Data Map:
Collect and access Hologres metadata.
Use the metadata acquisition feature to import metadata from a Hologres data source to Data Map for centralized management. For more information, see Metadata acquisition.
After the metadata acquisition is complete, go to the Overview page to view statistics and table information for Hologres databases that have a metadata collector configured in the current region. For more information, see Data overview.
To find a table, see Query and manage common data.
View lineage details.
Find the target table and click its name to go to the table details page. On this page, you can view basic information, output information, and lineage information. For more information, see View table details.
The following figure shows an example of a Hologres internal table. You can view its metadata on this page and its lineage on the Lineage tab.
View the lineage with MaxCompute
DataWorks Data Map also supports viewing the data lineage between MaxCompute and Hologres. For example, you can click the Hologres foreign table link to view the corresponding MaxCompute table information.
To navigate to the MaxCompute table details page from the foreign table link, the project that contains the MaxCompute table must be attached to the target DataWorks workspace.
You can view the table lineage. For example, the following figure shows that data is written to a MaxCompute table through a Hologres foreign table in the scheduling module of DataWorks. You can also go to the Field Lineage tab to view the field lineage.
View the lineage with Flink
You can view the lineage between Hologres and Flink in the Flink console. For more information, see View data lineage.