DataWorks:Manage Open Data

View Open Data

You can view the following information to understand the metadata and scenarios of different packages in Open Data. This helps you select and use packages as needed.

1. Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Governance > Data Map. On the page that appears, click Go to Data Map.

2. In the navigation pane on the left of the Data Map page, click the icon to go to the Data Catalog page.

3. In the catalog list, click DataWorks OpenData.

4. Switch to the Package List tab. Click Details for a package. On the Table List tab, you can view the metadata tables and their descriptions in the package.

   • Differences between MetaData and Examples metadata:

     Package name	Open Data	Scenarios
     MetaData	Table information, such as table schema, field descriptions, and data lineage. Scheduling nodes and instances, such as task execution status and dependencies. Management metadata, such as workspaces, members, and projects. Metadata, such as Data Quality details and data governance details.	Used for data asset inventory, data lineage analysis, and dependency management.
     Examples	A collection of metadata metrics. Note The Examples package is provided by DataWorks and contains a collection of sample metric tables generated from metadata. The actual data varies as needed. The data displayed in the UI prevails.	Combined with the raw data from the MetaData package, you can quickly build common analysis scenarios, such as resource utilization analysis and task health assessment.

     Important

     For more information, see Details of Open Data table schemas.

5. To learn more about a metadata table, click Details for the table. On the table details page, you can view the Field Name and their Description. This helps you understand the relationships between Open Data metadata tables.

Install Open Data

You can install all relevant data views in your tenant as a package. This action grants authorization to the MaxCompute compute engine of a specified destination workspace.

1. Go to the Open Data page. On the Package List tab, select the target package (MetaData or Examples) and click Details in the Actions column.

2. On the package page, install the package that you want to use.

   • First-time installation: In the upper-right corner of the package page, click Installation.

   • If an installation record exists: On the Installation History tab of the package page, click Installation.

   

3. In the Install DataWorks Metadata dialog box, select the destination workspace and the MaxCompute project to which you want to grant permissions.

   Note

   • The MaxCompute project to which you grant permissions is the compute engine that is bound to the destination workspace.

   • If the MaxCompute compute engine that is bound to the destination workspace is in the Installed state, you do not need to install it again.

   • To use the metadata in Open Data in a workspace that is in standard mode, we recommend that you install and grant permissions for the MaxCompute compute engines that are bound to the development and production environments.

4. Read the installation notes, select the confirmation checkbox, and then click Confirm Installation. After the installation is successful, you can view the new installation record on the Installation History tab.

To use metadata from other packages in the MaxCompute compute engine of the destination workspace, follow the steps above to install and grant permissions.

Use Open Data

After you install Open Data to the MaxCompute compute engine of the destination workspace, you can directly access the authorized metadata views in the Data Development or DataAnalysis module using the MaxCompute compute engine.

1. In the navigation pane on the left of the Data Map page, click the icon to go to the Data Catalog page.

2. In the catalog list, click MaxCompute, and then click the package that is prefixed with u_meta and suffixed with the region name.

   

3. On the details page, choose Use Now > Data Studio or Use Now > Data Analysis. You are redirected to the corresponding module to query and use the authorized metadata views.

   • Use in Data Studio:

     1. In the top menu bar of Data Studio, switch to the region and workspace where Open Data is installed.

     2. Create a MaxCompute node. On the node editing page, you can use Open Data to develop tasks.

     3. You can use the following sample code to verify and test whether the installation is successful.

        SELECT  dt
                ,COUNT(*) AS database_count
        FROM    u_meta_hangzhou.databases
        GROUP BY dt
        ORDER BY dt ASC
        LIMIT   32
        ;

        Important

        • When you test the code, you must use the authorized MaxCompute compute engine.

        • Replace u_meta_hangzhou with the name of the package in your MaxCompute data catalog. The package name is prefixed with u_meta and suffixed with the region name.

   • Use in DataAnalysis:

     1. After you go to the SQL query page of DataAnalysis, the system provides sample metadata analysis scripts. You can modify the scripts based on your requirements.

     2. Click the icon in the upper-right corner of the SQL query page, select the authorized workspace, and bind the data source that has the same name as the one generated by the system when you bound the MaxCompute compute engine. Then, you can query the authorized metadata in DataAnalysis.

        Note

        Before you run the sample script, replace the REPLACE_WITH_WORKSPACE_ID parameter with the ID of the workspace that you want to query. Otherwise, an error is reported. For information about how to obtain a workspace ID, see Configure a workspace.

Uninstall Open Data

If you no longer need to use Open Data or need to revoke the authorization of a MaxCompute project in a workspace, you can uninstall Open Data.

1. In the navigation pane on the left of the Data Map page, click the icon to go to the data catalog page.

2. In the catalog list, select MaxCompute and then click the package that is prefixed with u_meta and suffixed with the region name.

3. On the Accessible Projects tab of the details page, find the destination project and click Uninstall in the Actions column. In the Confirm Uninstall DataWorks Metadata dialog box, read the uninstallation notes, select the confirmation checkbox, and then click Confirm Uninstall.

   Important

   Uninstalling revokes permissions and breaks tasks that depend on this data. Verify that no active dependencies exist before you proceed.