After you attach a Lindorm instance to DataWorks, you can manage and view its metadata in Data Map. This topic describes how to view and manage data in Lindorm tables.
Prerequisites
You can set the Spark parameters on the computing resource attachment page to collect metadata from Lindorm. For more information, see Set Spark parameters for a Lindorm computing resource.
Go to Data Map
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, click Go to Data Map.
Collect metadata of Lindorm tables
Click the 
icon in the navigation pane on the left of the Data Map page to go to the Database Collector management page. In the Data Source Perspective section, find the Lindorm collector and configure a metadata collection plan to collect metadata from Lindorm tables. For more information, see Metadata acquisition.
Lookup table
After the metadata collection plan for Lindorm runs successfully, you can follow these steps to view the collected metadata of the Lindorm tables.
Click the
icon in the left-side menu bar to go to the search page.In the Type directory tree on the left, select Table. Under Data Source, find and click the Lindorm data source to go to the Lindorm table list page.
NoteIf multiple data sources exist, the list may not be fully displayed. The More button appears to the right of Data Lake Formation. Click More to find the Lindorm data source.
In the search box above the Lindorm table list, enter a keyword from the table name to find the table.
View table details
On the Lindorm table list page, click the Lindorm table that you want to view. The table details page appears.
Feature | Description | References |
Shortcut operations | At the top of the page, you can Add To Album, View Added Albums, or Add To Favorites. | |
Table Basic Information | In the Table Basic Information section, view information such as the Database, Table Type, Owner, Creation Time, and whether it is a partitioned table. | |
Table Technical Information | View information such as the table storage Path, Input Format, and Output Format. | |
Details | View the Field Information and field lineage of the table. | |
Lineage | View the data lineage between Lindorm tables. | |
Usage Notes | You can perform the Edit and View Historical Versions operations to learn relevant information from the business description of the data. |
Manage table favorites
On the table details page, you can follow these steps to add a table to or remove a table from your favorites for centralized management and quick access.
Add to/Remove from favorites.
Add to favorites: Click the Add To Favorites button below the table name to add the table to your favorites.
Remove from favorites: Click the Remove From Favorites button below the table name to remove the table from your favorites.
View your favorite tables.
Click the
icon in the navigation pane on the left of the Data Map page. In the directory tree on the left, choose to go to the My Favorites page. Set Data Type to Lindorm to view your favorite Lindorm tables.
View basic information about a table
In the left part of the table details page, you can view the Table Basic Information, Technical Information, and Business Information sections to better understand the table.
Table Basic Information: Includes information such as Table Name, Created At, and Partitioned Table.
Technical Information: Includes information such as the storage path and input/output formats.
Business Information: Includes information such as the Region Of The Workspace and the Instance.
View table details
Click the Details tab on the table details page to view detailed information about the table.
View field information.
You can view the name and data type of each field in the table.
Search for fields.
If the table contains many fields, you can enter a field name in the search box under Field Information to quickly locate the field.
Preview field descriptions.
Click Recommended Field Desc under Field Information. Copilot then quickly generates field descriptions to help you better understand the meaning and purpose of the fields.
View field lineage.
Click the
icon in the Actions column for a specific field to view its data lineage and understand its relationships with other fields or tables.
View table lineage information
Data lineage shows the relationships between tables and fields. It is derived from the actual data flow, such as parsed scheduling jobs and data synchronization. On the lineage page, you can view the upstream and downstream nodes of a table or field. You can also view the source of the raw data and the destination database of the table data. As needed, you can also analyze the impact of lineage at different levels.
The data lineage feature is available only in DataWorks Standard Edition and later versions.
Table and field lineage is derived from the actual data flow, such as parsed scheduling jobs, to show the relationships between different tables and fields. This is a real-time statistical operation.
If Data Map cannot properly display the data lineage generated by running SQL statements in a Lindorm Spark SQL node, you can manually set the relevant DataWorks scheduling parameters in the code of the Lindorm Spark SQL job. For more information, see Attach a Lindorm computing resource. The following SQL statements can generate data lineage:
insert into /insert overwrite table
create as select from table
create table
create table like
Click the Lineage tab on the table details page. You can view Table Lineage and Field Lineage information to manage and understand the lineage relationships between tables and fields.
View table lineage
View basic information: Hover the mouse pointer over a table or node type to view details.
View the number of upstream and downstream nodes: In the table lineage graph, you can view the number of upstream and downstream nodes for the current table.
Expand or collapse nodes: As needed, you can click the
or 
icon in the lineage graph to show or hide the corresponding upstream or downstream nodes.
View field lineage
Switch fields: In the Switch Field area, you can select a different field from the current table to view its lineage graph.
View basic information: Hover the mouse pointer over a field or node type to view details.
View the number of upstream and downstream nodes: In the field lineage graph, you can view the number of upstream and downstream nodes for the current field.
Expand or collapse nodes: You can click the
or icon in the lineage graph to show or hide the corresponding upstream or downstream nodes.
View usage notes for a table
Click the Usage Notes tab on the table details page. You can Edit the notes or View Versions to manage and understand the application scenarios of the table at different times.
Edit usage notes for the table.
Click Edit. In the dialog box that appears, enter the usage notes for the current table and click Save.
View historical versions of the usage notes.
Click View Versions to see all version records for the table. You can also click View in the Actions column to understand the application scenarios of the table in different time periods.