Data Map is a module for managing enterprise data catalogs based on metadata. It includes features such as global data retrieval, viewing metadata details, data preview, data lineage, and data category management. Data Map helps you find, understand, and use data more effectively.
Version limits
Code search and data lineage analysis are supported only in DataWorks Standard Edition or later.
Data albums are supported only in DataWorks Professional Edition or later.
Metadata acquisition
MaxCompute and DLF metadata: The system has built-in metadata collectors for MaxCompute and DLF. Data Map automatically collects the metadata.
Other types of metadata: You can use the custom metadata collection feature to import metadata from various data sources into Data Map for unified management. Data Map supports the following data source types: Hologres, EMR Hive, CDH Hive, StarRocks, AnalyticDB for MySQL, AnalyticDB for PostgreSQL, MySQL, PostgreSQL, Oracle, SQL Server, ClickHouse, Elasticsearch, Tablestore (OTS), Lindorm, and OSS. For more information, see Metadata acquisition.
Network connectivity
To import metadata from a data source into Data Map for unified management, you must ensure that the Data Map metadata collector can access your database. If the source database uses a whitelist for access control, you must add the IP address blocks of the region where your DataWorks workspace is located to the database's whitelist. For more information about the whitelists that you need to configure, see Configure IP address whitelists for a data source that is used for metadata acquisition.
Data overview
On the Overview page, you can view the following information about MaxCompute in the current region: the total number of projects, tables, and APIs, total storage, a storage trend chart, top projects by storage, top tables by storage, and popular tables.
After metadata is acquired, you can view information such as the total number of databases, tables, and APIs for data sources such as AnalyticDB for MySQL 3.0, MySQL, Hologres, AnalyticDB for PostgreSQL, and Tablestore.
For more information, see Data overview.
Metadata retrieval
Data Map collects metadata for objects such as tables, code, indexes, APIs, datasets, AI models, algorithm tasks, model services, and workspaces. You can search for these objects within the current region using filter conditions such as type, workspace, owner, environment, tag, and category. For more information, see Query and manage common data.
You can also view tables and datasets from data sources in a data catalog format.
Metadata details
After you find a target object in Data Map, you can click the object name to open its details page.
For tables, you can view details such as Basic Information, Output Information, and Lineage Information. For more information, see View table details.
NoteOn the Lineage tab, you can view the auto-captured lineage of engine nodes. The product page provides details on lineage support for specific engine nodes. Additionally, you can view the end-to-end lineage of a MaxCompute table used for batch synchronization, including its ancestor and descendant tables. By expanding the table lineage levels, you can view information about the table's original data sources and destination databases.
For APIs, you can view information such as parameter information, response examples, lineage information, and instructions. For more information, see View the details of an API.
For datasets, you can view information such as property information, lineage information, and instructions.
You can also retrieve and view code. You can click Details to navigate to the source, such as a node or resource. For more information about metadata details, see Find and process data.
Table organization and management
The category management feature helps you efficiently organize and manage tables by category. After you configure categories for tables, you can filter for specific tables by category when you search for them. For more information, see Configuration management.
An Alibaba Cloud account or a Resource Access Management (RAM) user who has the AliyunDataWorksFullAccess permission can edit the category tree.
Data albums: DataWorks lets you organize data related to specific subjects into a data album that is managed from a business perspective. This helps you view and manage data.
Category management
Add tables to a specified category in batches on the page.
After you configure category navigation, you can select the last-level category. Then, you can use the quick add entry on the interface to add multiple tables from a project to the category in batches. For more information, see Configuration management.
You can add tables in batches to a specified category on the My Data page.
After you configure categories, you can go to the My Data page (Owned by Me or Managed by Me) to add tables to a specified category in batches. For more information, see My Data.
Table management: For MaxCompute data, Data Map lets you modify display names and lifecycles in batches, delete development or production tables in batches, and change table owners in batches. For more information, see My Data.
Favorites: Data Map lets you add tables that you follow to your favorites. This provides centralized management and helps you quickly locate and view the tables. You can view your favorited tables in the My Favorites group on the My Data page. For more information, see My Data.
Add a table to My Favorites: On a table's details page, click the favorite button to add the table to My Favorites. For more information, see View the details of a table. After you add the table to favorites, you can view it in the My Favorites group on the My Data page. For more information, see Favorite a table.
To remove a table from the My Favorites list:
On the My Data page, you can remove a table from the My Favorites group.
On the details page of a favorite table, click Remove From Favorites to remove the table from your favorites.
Table permission control
Request table operation permissions
By default, RAM users in a standard mode workspace cannot use SQL commands to directly operate on production tables. To operate on production tables or query them across accounts, you must request permissions. To request the required permissions, go to the table details page and click Request Permissions. You are then redirected to Security Center to complete the request. For more information, see Request permissions on a table.
NoteIf a RAM user does not have query permissions for a table, the user cannot preview the table's data on the table details page in Data Map by default.
Control data preview permissions for MaxCompute tables
On the Project Management Configuration tab of the Configuration Management page, you can enable the preview permission for MaxCompute development or production tables in a specific workspace. After this permission is enabled, all members in the workspace can preview the tables in the project without requesting access permissions. For more information, see Project management configuration.
NoteThis operation may cause sensitive data leakage. Evaluate the risks and proceed with caution.
The project owner or workspace administrator can perform this operation.
This permission only controls the Data Preview feature on the table details page in Data Map.
Hide tables: After a table is hidden, it will not appear in search results. You can hide the table from everyone or make it visible only to users in the workspace where the table resides. For more information, see My Data.
Hide a single table: On the Owned by Me tab of the My Data page, update the table's visibility.
NoteBy default, table owners and workspace administrators are not affected by this visibility setting.
Batch-hide tables at the project level: On the Project Management Configuration tab of the Configuration Management page, you can select a Project and set the hide property for all tables in the project.
Refresh table metadata
If a table exists but cannot be found in a search, or if a table is updated but the changes do not appear in Data Map, you need to refresh the table metadata.
On the search results page, click Refresh Table Metadata.
Go to the page in Data Map:
After you complete one of these operations, you can search for the table again in Data Map.