DataWorks:Data Map Overview

Limitations

• Code search and data lineage analysis are available only in DataWorks Standard Edition and higher.

• Data albums are available only in DataWorks Professional Edition and higher.

Collect metadata

• MaxCompute metadata & DLF metadata: Data Map automatically collects MaxCompute and DLF metadata through built-in collectors.

• Other types of metadata: Use custom metadata collection to import metadata from other data sources, including Hologres, E-MapReduce, CDH Hive, StarRocks, AnalyticDB for MySQL, AnalyticDB for PostgreSQL, MySQL, PostgreSQL, Oracle, SQL Server, ClickHouse, Elasticsearch, Tablestore, Lindorm, and OSS. Collect metadata. Metadata collection for Hudi is not supported.

Network connectivity

To import metadata, ensure that the Data Map metadata collector can access your database. If your database uses a whitelist for access control, add the CIDR blocks for your DataWorks workspace region to the whitelist. Whitelist configurations required for metadata collection from data sources with access control.

Overview

• The Overview page shows MaxCompute resource metrics for the current region: project count, table count, storage usage, API count, storage trends, top projects by storage, top tables by storage, and popular tables.

• View database, table, and API counts for data sources such as AnalyticDB for MySQL 3.0, MySQL, Hologres, AnalyticDB for PostgreSQL, and OTS after metadata collection completes.

Overview.

Metadata retrieval

Data Map aggregates metadata for tables, code, indexes, APIs, datasets, AI models, algorithm tasks, model services, and workspaces. Search within the current region by type, workspace, owner, environment, tag, and category. Metadata retrieval.

On the Data Map search page, the left panel lets you filter by data source (MaxCompute, EMR Hive, Data Lake Formation, Oracle, Hologres, and more). Search supports three modes: Tables only, Columns only, and Tables and columns. In the search results, you can Request Permissions or View Lineage for each table.

You can also browse tables and datasets in a data directory format.

Metadata details

Click an object name to open its details page.

• For tables, view Basic Information, Output information, and Lineage Information. View table details.

  The Lineage Information tab contains the Table Lineage, Column Lineage, and Impact Analysis sub-tabs. On the Table Lineage sub-tab, click a table node in the lineage graph to view its entity type, name, data source type, storage size, lifecycle, and other properties in the right-side details panel. You can also click Request Permissions to submit a permission request.

  Note

  On the Data Lineage page, view the internal data lineage of engine nodes. Supported engine node types depend on the product page. MaxCompute also supports end-to-end lineage based on offline synchronization, letting you expand the lineage hierarchy to trace a table's upstream and downstream lineage to its original sources and final destinations.

• For an API, view Parameter Information, Sample Response, Lineage Information, and Usage Instructions. View API details.

  The API details page shows Basic Information (view count, call count, API ID, API path, workspace, owner, and more) and Technical Information (API type, request method, response type, data source type and name, table name) on the left side.

• For a dataset, view its Attribute Information, Lineage, and Usage Instructions.

  The Property Information tab displays the following:

  • Basic properties: Entity Type, Source, Data Type, Owner, Name, Workspace, Storage Type, Visibility, Created At, and Updated At.

  • Dataset version information: Version, URI, Bucket, Path, Region, and Default Mount Path. You can click View in OSS to navigate to the corresponding storage.

You can also search for code. Clicking a result redirects you to its source node or resource. metadata details.

Organize and manage tables

Use categories to organize tables and filter by category during search. Manage Configurations.

• Data album: Organize related data into a data album managed from a business perspective for easier viewing and management.

• Categories

  1. Batch add tables to a specified category from the Manage Configurations > Manage Categories page.

     After configuring category navigation, select a last-level category to batch-add tables from a specific project. Manage Configurations.

  2. Batch add tables to a specified category from the My Data page.

     Batch-add tables to a category from the My Data page (Owned by Me and Managed by Me). My Data.

• Table management: For MaxCompute tables, batch-modify display names, set lifecycles, delete development or production tables, or transfer ownership. My Data.

• Favorites: Add frequently accessed tables to Favorites. On the My Data page, view favorited tables in the My Favorites group. My Data.

  • Add a table to My Favorites: On the table details page, click Favorite to add the table to My Favorites. View table details. View the table on the My Data page under the My Favorites group. Add a table to favorites.

  • Remove a table from the My Favorites list:

    • On the My Data page, go to the My Favorites group and remove tables from favorites.

    • On the details page of a favorited table, use the Remove from Favorites entry to remove it.

Manage table permissions

• Request permissions to operate on tables

  In standard mode workspaces, RAM users cannot run SQL commands on production tables by default. To access production tables or query across accounts, click Apply for Permissions . This redirects you to Security Center . Request permissions on tables.

  Note

  By default, RAM users without table query permissions cannot use data preview on the table details page.

• Control data preview permissions for MaxCompute tables

  On the Manage Configurations > Manage Workspaces page, enable preview permissions for MaxCompute development or production tables in a specific workspace. All workspace members can then preview tables without requesting access. Manage Workspaces.

  Note

  • This operation may expose sensitive data. Evaluate the risks carefully before you proceed.

  • The project owner or workspace administrator of the workspace can perform this operation.

  • This permission controls only the Data Preview feature on the table details page in Data Map.

• Hide table: Hidden tables are excluded from search results. Hide a table from all users or restrict visibility to workspace members. My Data.

  • Hide a single table: On the My Data > Owned by Me page, update the table's visibility settings.

    Note

    By default, the table owner and workspace administrator are not subject to these permission controls.

    Click Update Table Info in the Actions column. In the dialog box, Visibility supports the following options:

    • Visible to all tenant members

    • Visible only to members of the associated workspace

    • Private

  • Bulk hiding at the project level: In Manage Configurations > Manage Workspaces, select Workspace, and configure the hiding properties for all tables in the project.

    In the MaxCompute Engine Configuration table, turn on the Default Project-only Visibility toggle to bulk-hide all tables under the corresponding compute engine.

Refresh table metadata

Refresh a table's metadata when it does not appear in search results or recent changes are not reflected.

1. On the search results page, click Refresh Table Metadata.

2. Go to the My Data > Refresh Table Metadata page in Data Map.

   Select a Data Type (MaxCompute, EMR Hive, or Data Lake Formation), enter the table GUID in the GUID field (for example, project.schema.table or odps.project.schema.table), and then click Refresh.

3. On the table details page of a MaxCompute or Data Lake Formation (DLF) table, click More > Refresh Metadata to manually refresh the table metadata. On a MaxCompute table, the More menu also provides a Generate API option, and the detail tabs include Data Insights, Data Health, and Usage History.

After refreshing, search for the table again in Data Map.