Data Catalog is a unified metadata workspace for data analysis. It brings together metadata from sources such as MaxCompute, Hologres, and DLF, allowing you to prepare tables and manage views without leaving DataWorks so you can focus on analysis.
Access the feature
Log on to DataWorks DataAnalysis, switch to the target region, and then click Go to DataAnalysis.
-
If Switch to New DataAnalysis appears in the navigation bar, click it to switch to the new version.
-
If Return to Legacy DataAnalysis appears in the navigation bar, you are in the new version.
Add a data catalog
If you have not used data analysis before, follow the instructions for new users. Otherwise, follow the instructions for existing users.
Add a data catalog as a new user
You can build a personalized catalog tree based on your needs to improve the experience. Follow these steps:
-
In Data Catalog, find the type of data source to add and click the
icon next to its name to open the Add Data Catalog page. -
On the Add Data Catalog page, find the instance or data source and click Operation in its Add column to add it to Data Catalog.
If you no longer need it, remove it.
Add a data catalog as an existing user
You can add system-recommended catalogs and data sources already added to a workspace to Data Catalog. After adding them, you can quickly view their tables, inspect table schemas, and generate query SQL.
Click the icon in the upper-right corner of the catalog.
-
DataMap - Metadata: Table metadata collected in Data Map. Each data source or compute resource can be added as a dataset.
-
DataMap - Data Album: data albums of thematic tables gathered in Data Map. Each data album can be added as a dataset.
-
My Favorites: Tables I have added to Favorites in Data Catalog.
-
My MaxCompute Tables: All MaxCompute tables for which the current account is the table owner.
-
Public Data: DataWorks provides the public datasets, which you can use to quickly generate test SQL statements for EMR Spark SQL, MaxCompute, and Hologres.
You can add up to 12 datasets. Remove any datasets you no longer need.
Manage Data Catalog
The following information applies only to Data Catalog for new users.
Use Data Catalog to manage data objects such as tables, views, external tables, resources, and functions.
-
MaxCompute: Manage a MaxCompute data catalog.
-
Hologres: Manage a Hologres data catalog.
-
DLF: Manage a DLF catalog.
Generate query SQL
Quickly generate query SQL from a table, then configure and run the query. For more information, see Create an SQL query.
-
Find the target data source in the Data Catalog list. Click the
icons on the left in sequence, and locate the table to query under Tables. -
Right-click the target table and select Generate SQL File for Query.
-
You can modify the generated query SQL script on the SQL editing page.
-
After making changes, click Run Configuration on the right side of the SQL editing page. Set Computing Resource, Script Parameters, and other information, and then run the query.
Appendix
MaxCompute authentication identity and authorization
To view MaxCompute data in Data Catalog as a RAM user or RAM role, first obtain the required MaxCompute permissions. If the added MaxCompute data source or project uses the three-layer model, also grant the RAM user or role permission to view Schema metadata.
If a MaxCompute project contains multiple Schemas, grant metadata permissions for all of them to view the complete Schema list on the project's details page in Data Catalog.
-
Authorize RAM users:
GRANT DESCRIBE ON SCHEMA <SchemaName> TO USER RAM$<AlibabaCloudAccountName>:<RAMUserName>; -
Authorize RAM roles:
GRANT DESCRIBE ON SCHEMA <SchemaName> TO USER `RAM$<AlibabaCloudAccountName>:role/<RAMRoleName>`;