DataWorks:Create an EMR table

Prerequisites

Before you begin, ensure that you have:

• An EMR computing resource configured in DataStudio. See Create an EMR computing resource

• Cluster metadata collected in Data Map. If no database appears after you add the EMR computing resource, re-run the crawler in Data Map. See Collect metadata from an EMR data source

Create the table

1. Go to the DataStudio page. Log on to the DataWorks console. In the top navigation bar, select a region. In the left-side navigation pane, choose Data Development and O\&M > Data Development. Select a workspace from the drop-down list and click Go to Data Development.

2. On the DataStudio page, hover over the icon and choose Create Table > EMR > Table. Alternatively, right-click EMR in the workflow where you want to create the table, and select Create Table.

3. In the Create Table dialog box, configure the parameters.

   

4. Click Create. The configuration tab for the table opens.

5. In the Basic attributes section, configure the parameters.

   Parameter	Description
   Level 1 theme	The name of the level-1 folder where the table resides. Level-1 and level-2 folders represent the table's location in DataWorks and help you manage tables.
   Level 2 theme	The name of the level-2 folder where the table resides.
   Create a theme	Click to open the Folder Management tab, where you can create level-1 and level-2 folders.
   Refresh	After you create a folder, click Refresh.
   Description	A description of the table.

   

6. In the Physical model design section, configure the parameters.

   Parameter	Description
   Layer	Select a level and business category from the drop-down lists. To create levels and business categories, click Create Level to open the Level Management tab. This action requires workspace administrator permissions. After creating entries, click Refresh.
   Physical classification	The physical classification of the table.
   Partition type	The partitioning strategy for the table. Valid values: Partition table and Non-partitioned table.
   Table type	Valid values: Internal tables and External tables.
   Select the storage format	Select a storage format for files in the table based on your business requirements.

   

7. In the Table structure design section, add fields and configure the table schema.

   Warning

   The following changes to an existing table require you to delete the table and recreate it with the same name. These operations are not allowed in the production environment: - Reordering fields (Move up / Move down) - Deleting a field - Adding a partition

   Parameter	Description
   Add fields	Click to add a new field. Configure the field information and click Save.
   Move up / Move down	Adjust the field order.
   Field name	The field name. Accepted characters: letters, digits, and underscores (_).
   Field type	The data type of the field. Supported types: TINYINT, SMALLINT, INT, BIGINT, FLOAT, DOUBLE, DECIMAL, VARCHAR, CHAR, STRING, BINARY, DATETIME, DATE, TIMESTAMP, BOOLEAN, ARRAY, MAP, and STRUCT.
   Length/Settings	The length limit for the field. Required for data types that enforce a length constraint.
   Description	A description of the field.
   Primary key	Marks the field as the primary key—a business concept that enforces record uniqueness. DataWorks does not enforce primary key constraints at the engine level.
   Edit	Click to edit the field, then click Save.
   Delete	Click to delete the field.
   Add (partition)	Available only when Partition type is set to Partition table. Click to add a partition field to the table.

   

8. Click the icon in the toolbar to commit the EMR table to the production environment.

   • Standard mode: Commit to the development environment first, then to the production environment.

   • Resource group: Select a resource group for scheduling. If you use a serverless resource group, DataWorks submits a table creation task to the compute engine and displays run logs. Use the run logs to troubleshoot any errors.

   If no serverless resource group is available, purchase and configure one. See Create and use a serverless resource group.