Data Catalog provides a unified interface for managing and working with Hive metadata. This topic describes how to create and manage table data objects in Data Catalog.
Open the Hive Data Catalog page
Go to the Workspaces page in the DataWorks console. In the top navigation bar, select a desired region. Find the desired workspace and choose in the Actions column.
In the left-side navigation pane, click the
icon. In the Data Directory catalog tree, click Hive to open the Hive Data Catalog management page.
Create a Hive Data Catalog
On the Hive Data Catalog management page, you can add an existing Hive data source as a dataset to the catalog list.
Click the
icon to the right of Hive Data Catalog to open the Add Instance page.On the DataWorks Data Source tab, add the Hive data source to the catalog list.
To manage EMR compute resources bound in the new Data Development interface in the current workspace, find the corresponding EMR cluster data source on the DataWorks Data Source tab. In the Operation column, click the Add button to add it.
Alternatively, on the DataWorks Data Source tab, select multiple Hive data sources and click the Batch Add button below the list to add them in a batch.
Manage the Hive Data Catalog
You can add and manage Hive tables in Hive Data Catalog.
Create a table
Click the
icon on the left side of Hive Data Catalog to find the Table.To the right of Table, click the
icon to open the Create Table page.You can generate the basic table information and field information in either of the following ways.
Create a table with Copilot:
In the toolbar at the top of the page, click Copilot table creation to open Copilot Chat.
Enter a table creation instruction in natural language, such as
Create a user table.Click Generate and Replace. The system generates a default table name and field information based on your instruction.
If the table name and fields meet your requirements, click Accept.
NoteIf you need to change some table information, click Accept and then manually edit the generated table information.
Create a table manually:
Create the table based on the parameter descriptions.
Parameter
Configuration description
Basic information
Specify the Table Name, Table Description, and other information.
Field information
Edit the fields and field annotations.
Manual editing: Above the field list, click the Insert button. After specifying the number of rows to insert, you can edit the Field Name, Data Type, and other information.
Intelligent editing with Copilot:Above the field list, click Generate Field or Generate field descriptions. Based on the table name and description you entered, the system generates related fields and descriptions.
Optional: Configure partition information.
To create a partitioned table, in the Partition Field area specify the number of partition field rows to add (multiple partitions are supported), and click Insert. Then, in the partition field list, configure the partition Field Name, Data Type and other related information.
Optional: Configure advanced settings .
Parameter
Configuration description
Table type
Only Internal Table.
Storage location
You can specify the table storage directory, such as
/user/hive/warehouse/hive_work.Storage format
Supported storage formats are CSV, PARQUET, ORC, AVRO, JSON, and SELE_DEFINE. Based on the selected format, the system automatically defines the data input and output formats and the serialization and deserialization methods.
CSV: A comma-delimited text file, suitable for simple data structures.
PARQUET: A columnar storage format with a high compression ratio, suitable for big data analytics.
ORC: An optimized columnar storage format with strong performance and support for complex data types.
AVRO: A binary format that supports schema evolution, suitable for dynamic data structures.
JSON: Supports nested structures and is suitable for semi-structured data.
SELE_DEFINE: Allows custom serialization and deserialization logic.
After configuration, click Publish in the top toolbar to create the table.
Manage tables
After creating a table in Hive Data Catalog, click the
icon on the left, then click Table to open the Table page.
View tables.
On the Table page, view the basic information of all tables. You can also click a table name to view its Details, Basic information and DDL information.
Delete a table.
On the Table page, find the table to delete. In the Operation column, click Delete.
ImportantDeleted tables cannot be recovered. Proceed with caution.
View or remove a Hive Data Catalog
If you no longer need a Hive Data Catalog, you can view or remove it.
View the data catalog.
After adding a Hive data source to Hive Data Catalog, click the
icon on the left to view the added Hive data sources.Click a Hive data source to view all Database information.
Unbind a project.
If you no longer need to manage a Hive Data Catalog, right-click it and select Disassociate Data Catalog.