Dataphin:Create metadata

Prerequisites

Ensure you have read permissions for the HBase, Elasticsearch, or MongoDB data sources before proceeding.

For more information, see request, renew, and return data source permissions.

Limits

• Each data table in HBase, Elasticsearch, or MongoDB can have only one set of metadata.

• Metadata creation is limited to users with developer or project administrator roles.

Create metadata

1. On the Dataphin home page, in the top menu bar, click Service > API Development.

2. In the upper-left corner, select a service project. In the left navigation pane, click Metadata Management, and then click the + New Metadata button.

   Create metadata for data tables in HBase data sources

   1. On the New Metadata page, configure the necessary parameters.

      Parameter	Description
      Data Table Configuration
      Physical Data Table	Select the data source type, data source, and data table to create metadata for HBase 0.9.4/1.1.x/2.x.
      Rowkey Configuration To read and write HBase data, locate the Region and its MemStore and HFile using the RowKey field.
      Separator	Choose a separator that matches the business data. If there is only one Rowkey field, select any separator. Available separators include DASH(-), AMPERSAND(&), CARET(^), SEMICOLON(:), POUND(#), VERTICAL(|), UNDERSCORE(_), ESCAPE_0(\0), ESCAPE_40(\40), ESCAPE_001(\001), ESCAPE_002(\002), ESCAPE_003(\003), ESCAPE_004(\004).
      New Rowkey Field	To create a Rowkey field, click Rowkey Configuration and select New Column Field. Configure the following: Rowkey Field Name: Customize the name following these rules: Include only letters, numbers, or underscores (_). Begin with a letter. Limit to 64 characters. Field Type: Choose a Rowkey field type that matches the business data. Supported field types encompass Double, Float, String, Date, Boolean, Int, Long, Short, Byte, Bigdecimal, and Binary. For business data with field types not listed, it is recommended to use String. Rowkey Description: Provide a brief description of the Rowkey field.
      Column Field Configuration
      New Column Field	Column Family Name: Match it to the actual column family name in the HBase data source. Column Field Name: Match it to the actual column field name in the HBase data source. Field Type: Choose the corresponding field type based on the business data. Supported field types encompass Double, Float, String, Date, Boolean, Int, Long, Short, Byte, Bigdecimal, and Binary. For business data with field types not listed, it is recommended to use String. Description: Provide a brief description of the column field.

   2. Click Submit to finalize the creation of HBase metadata.

   Create metadata for data tables in Elasticsearch data sources

   1. On the New Metadata page, configure the parameters.

      Parameter	Description
      Data Table Configuration
      Physical Data Table	Select the data source type, data source, and data table to create metadata for Elasticsearch.
      Column Field Configuration
      New Column Field	Column Name: Configure it as the actual field name in the business data. Multilayer nesting is supported, using a dot (.) as a separator. Field Type: Select the corresponding field type based on the business data. Supported field types encompass Double, Float, String, Date, Boolean, Int, Long, Short, Byte, Bigdecimal, and Binary. For business data with field types outside this range, it is recommended to use String. Description: Provide a brief description of the column field.

   2. Click Submit to finalize the creation of Elasticsearch metadata.

   Create metadata for data tables in MongoDB data sources

   1. On the New Metadata page, configure the parameters.

      Parameter	Description
      Data Table Configuration
      Physical Data Table	Select the data source type, data source, and data table to create metadata for MongoDB.
      Column Field Configuration
      New Column Field	Column Name: Configure it as the actual field name in the business data. Multilayer nesting is supported, using a period (.) as a separator. Field Type: Select the appropriate field type based on the business data. Supported field types encompass Double, Float, String, Date, Boolean, Int, Long, Short, Byte, Bigdecimal, and Binary. For business data with field types not listed, it is recommended to use String. Description: Provide a concise description of the column field.

   2. Click Submit to complete the creation of MongoDB metadata.

Manage metadata

Once the metadata is created, you can manage it by editing or deleting on the Metadata Management page.