All Products
Search

This Product

    Dataphin:Create TDH Inceptor Data Source

    Document Center

    Dataphin:Create TDH Inceptor Data Source

    Last Updated:Mar 05, 2025

    Creating a TDH Inceptor data source allows Dataphin to read and write business data from and to TDH Inceptor. This topic explains the process of establishing a TDH Inceptor data source.

    Limits

    Only custom global roles with the create data source permission point, super administrator, data source administrator, section architect, and project administrator roles can create data sources.

    Procedure

    1. On the Dataphin home page, select Management Center > Datasource Management from the top menu bar.

    2. On the Datasource page, click + New Data Source .

    3. In the New Data Source dialog box, select Big Data area, then choose TDH Inceptor .

      If you have recently used TDH Inceptor, it can also be selected from the Recently Used area. Alternatively, enter the keyword for TDH Inceptor in the search box to quickly filter your options.

    4. In the New TDH Inceptor Data Source dialog box, enter the basic information for the data source.

      Parameter

      Description

      Datasource Name

      The name must meet the following conventions:

      • Can only contain Chinese characters, uppercase and lowercase English letters, numbers, underscores (_), or hyphens (-).

      • Cannot exceed 64 characters in length.

      Datasource Code

      After configuring the data source code, you can reference tables in the data source in Flink_SQL tasks using the format datasource code.table name or datasource code.schema.table name. If you need to automatically access the data source corresponding to the environment, use the variable format ${datasource code}.table or ${datasource code}.schema.table. For more information, see the referenced document.

      Important

      • Once the data source encoding is successfully configured, it cannot be modified.

      • After the data source encoding is successfully configured, data preview can be performed on the object details page of the asset directory and asset checklist.

      • In Flink SQL, currently only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, and SelectDB data sources are supported.

      Version

      You can choose 6.2.x or 9.3.x.

      Data Source Description

      A brief description of the data source. Must not exceed 128 characters.

      Datasource Config

      Select the data source to configure:

      • If the data source distinguishes between production and development data sources, select Production + Development Data Source.

      • If the data source does not distinguish between production and development data sources, select Production Data Source.

      Tag

      You can classify and tag the data source according to the tag. For information on how to create a tag, see the referenced document.

    5. Set up the connection parameters between the data source and Dataphin.

      If your data source configuration selects Production + Development Data Source, you need to configure the connection information for Production + Development Data Source. If your data source configuration is Production Data Source, you only need to configure the connection information for Production Data Source.

      Note

      Typically, production and development data sources should be configured separately to maintain environment fencing and minimize the impact of development activities on production data sources. However, Dataphin also allows for them to be configured with the same parameter values.

      Parameter

      Description

      Cluster Configuration

      NameNode

      NameNode is the HostName or IP and port of the NameNode node in the HDFS cluster.

      Configuration example: host=192.x.x.169,webUiPort=,ipcPort=8020. Among them, webUiPort and IPCport are 50070 and 8020 by default in the TDH environment. You can fill in the corresponding ports according to the actual situation.

      Configuration File

      Used to upload Hadoop configuration files, such as hdfs-site.xml and core-site.xml. The configuration files can be exported from the Hadoop cluster.

      Authentication Type

      If the HDFS cluster does not require authentication, select No Authentication. If the HDFS cluster requires authentication, Dataphin supports selecting Kerberos.

      If you select the Kerberos authentication method, you need to configure the relevant authentication information:

      • Kerberos Configuration Method:

        • KDDC Server: Unified service endpoint for KDC, supporting multiple configurations separated by semicolons (;).

        • Krb5 File Configuration: You need to upload the Krb5 file.

      • HDFS Configuration:

        • HDFS keytab File: The keytab file for HDFS, which is the file for Kerberos authentication.

        • HDFS Principal: The Principal name for Kerberos authentication. An example is xxxx/hadoopclient@xxx.xxx.

      Inceptor Configuration

      JDBC URL

      Configure the JDBC URL for connecting to Inceptor, in the format jdbc:hive2//host:port/dbname.

      Authentication Type

      If the Inceptor cluster does not require authentication, select No Authentication. If the Inceptor cluster requires authentication, Dataphin supports selecting LDAP or Kerberos. You can choose according to the actual situation. The detailed content is as follows:

      • Kerberos: After selecting, you need to upload the Keytab File and configure the Principal. The Keytab File is the file for Kerberos authentication. An example of the Principal is xxxx/hadoopclient@xxx.xxx.

      • LDAP: After selecting, you need to configure the username and password for LDAP authentication.

      Username

      Configure the username for Inceptor.

      Metadatabase Configuration

      Metadata Retrieval Method

      Supports Metadatabase method and HMS method for retrieval.

      • Metadatabase Method: Requires configuring the database type, version, JDBC URL, authentication method, username, and password.

        • Database Type: Select the database type according to the metadatabase type used in the cluster. Supports selecting MySQL, PostgreSQL, Inceptor.

        • Version: If the database type is MySQL, you need to select the corresponding version. Supports MySQL5.1.43, MySQL5.6/5.7, and MySQL8.0 versions.

        • JDBC URL: Fill in the link address of the corresponding metadatabase.

          • MySQL: The format is jdbc:mysql://host:port/dbname.

          • PostgreSQL: The format is jdbc:postgresql://host:port/dbname.

          • Inceptor: The format is: jdbc:hive2://host:port/dbname.

        • Authentication Method: If the database type is Inceptor, you need to fill in the authentication method. Supports three authentication methods: No Authentication, LDAP, Kerberos.

          When using the Kerberos authentication method, you need to select the authentication method as Kerberos in the cluster configuration area.

        • Username, Password: The username and password for logging on to the metadatabase.

      • HMS Method: Requires configuring the Authentication Method and uploading hive-site.xml.

        • Authentication Method: Supports No Authentication, LDAP, Kerberos.

          When using the Kerberos authentication method, you need to select the authentication method as Kerberos in the cluster configuration area. You also need to upload the Keytab File and fill in the Principal.

        • hive-site.xml: Upload the hive-site.xml configuration file for the TDH Inceptor data source cluster.

    6. Click Test Connection to verify that the data source can communicate properly with Dataphin.

    7. Once the test is successful, click OK to finalize the creation of the TDH Inceptor data source.