How to find explore and understand data assets - Data Map - Alibaba Cloud - DataWorks

DataWorks Data Map is a centralized platform for managing and discovering data assets. It helps you gain a comprehensive view of your enterprise data landscape and find the data you need quickly and accurately, just like using a search engine. Its core features include:

Data Overview: Provides statistics about your data assets for a quick overview of their current state.
Global keyword search: Allows you to quickly locate tables, fields, or business terms of interest from a single entry point.
Multi-dimensional filtering and browsing: Enables refined and exploratory data discovery by combining various conditions.

Limits

Tenants with an existing PAI workspace can search for and view related AI assets, such as datasets, AI models, algorithm tasks, and model services.
Code search is available only to users of DataWorks Standard Edition or later.

Access Data Map

Log on to the DataWorks console. In the target region, click Data Governance > Data Map in the left-side navigation pane. On the page that appears, click Go to Data Map.

Data overview

The Data Overview page provides a high-level summary of how your data sources are used.

On the Data Map Homepage, click Data Overview next to Metadata Collection.

On the Data Overview page, view all engine resources in the current region.

MaxCompute key metrics:

Name	Description
Total Number of Projects	The total number of MaxCompute projects in the current region. This data is updated in near-real-time.
Total Tables	The total number of MaxCompute tables in the current region. This data is collected offline and has a T+1 latency.
Storage	The total logical storage size of all tables in the current region. This includes temporary files from scheduled tasks and storage space that is not yet released after tables are deleted. This data is collected offline and has a T+1 latency.
Total APIs	The total number of MaxCompute-type APIs that are published to API Gateway in the current region.
Storage Trend Chart	A trend chart showing the total logical storage of MaxCompute projects in the current region. This includes temporary files from scheduled tasks and storage space that is not yet released after tables are deleted. This data is collected offline and has a T+1 latency.
Top projects by storage	A ranking of MaxCompute projects by logical storage size in the current region. This data is collected offline and has a T+1 latency. You can click a project to view its metadata. Important Project storage includes not only table storage but also resource storage, recycle bin storage, and other system file storage, so it is larger than table storage alone.
Top tables by storage	A ranking of MaxCompute tables by size. This data is collected offline and has a T+1 latency. You can click a table to view its metadata. Important Table storage is calculated based on logical storage rather than physical storage.
Most Frequently Viewed Tables	The most frequently viewed tables in the last 30 days, ranked by page views (PV) on the table details page in Data Map. This data is updated in near-real-time.

Other engine metrics: Displays information such as the total number of databases, tables, and clusters.

Find data

Data Map provides two complementary search methods to meet your different data discovery needs.

Global keyword search

When you have a clear idea of the data you are looking for, the global keyword search is the most efficient method. The homepage also provides quick access to Recently Viewed, Followed Collections, and Managed Collections.

Go to the search bar: The search bar is at the top of the Data Map homepage.
Enter keywords: Select a metadata type and enter a Keyword, such as a table name, field name, or comment. For example, enter "user", "order", or "user_info".
Run the search: Press Enter. The system returns a list of all matching data assets. Click an asset to view its details.

Multi-dimensional filtering and browsing

When you need to narrow your search or explore by business category, you can perform an advanced search on the search page. The global keyword search bar remains available at the top of the page.

Go to the search page: Click the search icon in the left-side navigation pane to open the advanced search page.
Combine filter conditions: On the left side of the page, select different filter conditions to progressively narrow down the results:
- Type: Select a metadata type. Supported types include table, code, Index, API, dataset, Workspace, and more.
- Data source: When you select Table, you can limit the search to a specific engine, such as MaxCompute, EMR Hive, or Hologres.
  
  Important
  If the number of data sources exceeds the display limit, click the More button in the upper-right corner to show hidden data sources.
- Filter conditions: You can filter by project, owner, environment, cluster, Data Catalog, database, and more.
  
  The available filter conditions vary depending on the metadata type.
  Note
  In addition to basic attributes, table objects can be filtered by tags and categories, while workspaces and datasets support filtering by custom attributes.
  - Tags: You can filter by tags applied during data governance. For more information, see Tags.
  - Category You can filter by business category. For more information, see Category configuration.
  - Custom attributes: You can filter by enumeration-type custom attributes. For more information, see Custom attributes.

Example:

I want to find all tables in the production environment of MaxCompute that are owned by "John Doe".Action: In "Data source", select MaxCompute → In "Owner", enter John Doe → In "Environment", select Production.

AI Search

When it is hard to describe what you are looking for with a single keyword, or when you want suggestions based on a business scenario, use AI Search to locate assets, recommend related tables, and explain the rationale through natural-language conversation.

From the search page: At the upper-right of the Data Map search page, switch to the AI Search tab to enter the conversational search experience.
From the home page: The search box on the Data Map home page can be switched to the smart-question mode. After you submit a question, the system jumps to the AI search session page to continue the conversation.

How to use: Describe your need in natural language in the input box, for example:

Find fact tables related to the trading business; query datasets added in the last 7 days; list MaxCompute projects that belong to the "Data Governance" business unit.

Session management: A question raised from the home page creates a new session. When you open the AI search page directly, your last session is continued by default if no specific session is specified. You can create a new session or return to a history session as needed.

Quick question templates: Common question templates are provided at the bottom of the AI search session page. Click a template to send it directly.
Relationship with global search: Global search matches metadata exactly by keyword and filters; AI search understands intent by natural language and semantic context and provides recommendations. The two complement each other—use global search when you know the target object, and use AI search when you describe a business need.

Note

AI search is currently rolled out by tenant gray release. It is visible only when your tenant has the capability enabled. To request it, contact your administrator or submit a ticket.

FAQ

Q: Why can't I find the table I'm looking for?

A: If you cannot find a table in DataWorks Data Map, check the following potential causes and solutions.
- Search keyword mismatch: The search supports retrieval by table name, description, and field name. Try using a more precise or partial keyword.
- Metadata not synchronized: DataWorks automatically collects metadata after you associate it with MaxCompute or DLF, but there may be a synchronization delay after you create or modify a table.
  Solution: Go to My Data > My Tools > Refresh Table Metadata. After the operation is successful, search for the table again.
- Table is hidden: The table may be set to Hidden, which makes it undiscoverable in searches. When a table's status is Hidden, it is invisible to everyone. When its status is Project Only, it is visible only to members of the workspace.
- Insufficient permissions: By default, a search covers only the current workspace and authorized projects. If the target table belongs to another account or an unauthorized workspace, it cannot be found. Additionally, if you do not have query permissions for a table, it is not visible in Data Map by default. This is especially true in standard-mode workspaces.
  
  Solution: Contact the table owner or an administrator to request permissions, or confirm that you are a member of the corresponding project.