DataWorks allows you to configure an Apache Gravitino data source to connect to the Gravitino unified metadata management service and collect and manage metadata from multiple heterogeneous data sources. As a metadata-type data source, it enables metadata collection and governance in Data Map after configuration. This topic describes how to configure this data source.
Overview
Apache Gravitino is an open-source unified metadata management framework that supports managing metadata from heterogeneous data sources such as Hive, Iceberg, and JDBC. With Gravitino, enterprises can achieve unified metadata management across engines and systems without changing the existing data storage architecture.
DataWorks provides the Apache Gravitino data source to integrate metadata managed by Gravitino into DataWorks Data Map. This enables metadata collection, discovery, and governance for data assets in Gravitino Catalogs, helping you build a unified view of your data assets.
Limits
-
Network connectivity: Only serverless resource groups are supported.
-
Use case: The Apache Gravitino data source is only applicable to Collect Metadata and governance. It cannot be used for data integration synchronization tasks.
-
Network requirements: The DataWorks resource group must be able to access the Gravitino service endpoint. Ensure network connectivity.
Procedure
1. Go to the data source page
-
Log on to the DataWorks console. Switch to the target region, click Workspace in the left-side navigation pane, and then click Manage in the Actions column of the target workspace.
-
In the Workspace Management Center, click Data Sources in the left-side navigation pane.
2. Create an Apache Gravitino data source
-
On the Data Sources page, click Add Data Source.
-
In the dialog box that appears, click the Metadata tab, and then search for and select Apache Gravitino.
3. Configure parameters
Configure the following parameters as prompted:
|
Parameter |
Description |
|
Data Source Name |
The custom name of the data source, such as |
|
Description |
The description of the data source for identification purposes. |
|
Configuration Mode |
The configuration mode of the data source. Currently, Selection Mode is supported, which allows you to fill in configuration items through the form. |
|
Endpoint |
The address of the Gravitino service, for example, |
4. Test connectivity
After configuring the data source, perform a connectivity test to ensure normal network communication between the data source and resource groups.
-
If Connected is displayed, the configuration is correct.
-
If Connection failed. is displayed, the system provides a diagnostic tool to help troubleshoot the issue. Common causes include incorrect endpoint configurations, network failures (IP whitelist not configured), and missing NAT gateways.
-
In standard mode, make sure that both the development and production environments show Connected. Otherwise, errors will occur during subsequent metadata collection.