Dataphin:Create a ClickHouse data source

Permissions

Only custom global roles with the New Data Source permission and the following system roles can create data sources: Super Administrator, Data Source Administrator, Domain Architect, and Project Administrator.

Procedure

1. On the Dataphin homepage, choose Management Center > Data Source Management in the top navigation bar.

2. On the Data Source page, click + New Data Source.

3. On the New Data Source page, in the relational database section, select ClickHouse.

   If you have used ClickHouse recently, you can also select it from the Recently Used section. You can also enter keywords in the search box to quickly find ClickHouse.

4. On the Create ClickHouse data source page, configure the connection parameters.

   1. Configure the basic information for the data source.

      Parameter	Description
      Data Source Name	Enter a name for the data source. Naming rules: Can contain only Chinese characters, uppercase and lowercase letters, digits, underscores (_), and hyphens (-). Can be up to 64 characters long.
      Data Source Code	After you configure the data source code, you can directly access tables in the Dataphin data source by using the Data_Source_Code.Table_Name or Data_Source_Code.Schema.Table_Name format in Flink SQL tasks or with a Dataphin JDBC client. To automatically switch data sources based on the task execution environment, use the ${Data_Source_Code}.table or ${Data_Source_Code}.schema.table variable format. For more information, see Develop with Dataphin data source tables. Important The data source code cannot be modified after it is configured. You can preview data on the object details page in the asset directory and asset checklist only after the data source code is configured. In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, SelectDB, and GaussDB data warehouse service (DWS) data sources are currently supported.
      Version	You can select v21 or v23. v23 is selected by default.
      Data Source Description	Enter a brief description of the data source. The description can be up to 128 characters long.
      Data Source Configuration	Select the type of data source to configure: If your business distinguishes between production and development data sources, select Production + Development Data Source. If your business does not distinguish between production and development data sources, select Production Data Source.
      Tag	You can add tags to classify the data source. For information about how to create a tag, see Manage Data Source Tags.

   2. Configure the connection parameters between the data source and Dataphin.

      If you select Production + Development Data Source, configure connection information for both. If you select Production Data Source, configure connection information for only the Production Data Source.

      Note

      Typically, the production data source and development data source should be configured as separate instances to isolate the environments and reduce the impact of development activities on production systems. However, Dataphin allows you to configure them as the same data source by using identical parameter values.

      For Configuration Method, you can select JDBC URL or host. JDBC URL is selected by default.

      JDBC URL

      Parameter	Description
      JDBC URL	The connection address for ClickHouse. The JDBC URL format is jdbc:clickhouse://host:port/dbname. If you are using ApsaraDB for ClickHouse, you can find the connection address and port information on the Cluster Information page in the ClickHouse console. After you connect to a cluster, you can run the show Databases command in the SQL Console to view the database name.
      Username, Password	The username and password for accessing the ClickHouse instance.

      Host

      • Host configuration

        Parameter	Description
        Server address	Enter the IP address and port number of the server. Click + Add to add multiple server addresses. Click the icon to delete an entry. At least one server address must be retained.
        dbname	Enter the database name.

      • Parameter configuration

        Parameter	Description
        Parameter	Parameter name: You can select an existing parameter name or enter a custom one. A custom parameter name can only contain uppercase and lowercase letters, digits, periods (.), underscores (_), and hyphens (-). Parameter value: This is required if a parameter name is selected. The value can contain only uppercase and lowercase letters, digits, periods (.), underscores (_), and hyphens (-), and can be up to 256 characters long. Note Click + Add Parameter to add more parameters, and click the icon to delete extra parameters. You can add up to 30 parameters.
        Username, Password	The username and password for accessing the ClickHouse instance.

      Note

      If you switch from the Host to the JDBC URL configuration method, the system automatically populates the JDBC URL field by concatenating the server addresses.

   3. Configure the advanced settings for the data source.

      Parameter	Description
      connection_timeout	The database connection timeout, in milliseconds. The default is 900,000 ms (15 minutes). Note If you include a connection_timeout setting in the JDBC URL, that value is used instead. For data sources created before Dataphin V3.11, the default connection_timeout is -1, which means there is no timeout.
      socket_timeout	The database socket timeout, in milliseconds. The default is 1,800,000 ms (30 minutes). Note If you include a socket_timeout setting in the JDBC URL, that value is used instead. For data sources created before Dataphin V3.11, the default socket_timeout is -1, which means there is no timeout.
      Connection Retries	If a connection times out, the system automatically retries it up to the specified number of times. The attempt fails if all retries are unsuccessful. Note The default is 1 retry. You can configure a value from 0 to 10. The connection retry setting applies by default to offline integration task and Global Quality (requires the asset quality feature to be enabled). You can configure a task-specific number of retries for offline integration tasks.

   Note

   For a duplicate parameter (a parameter defined in multiple locations), its value is determined by the following order of precedence:

   • If a parameter is defined in the JDBC URL, Advanced Settings, and Host Configuration, the value in the JDBC URL takes precedence.

   • If a parameter is defined in both the JDBC URL and Advanced Settings, the value in the JDBC URL takes precedence.

   • If a parameter is defined in both the Advanced Settings and the Host Configuration, the value in the Advanced Settings takes precedence.

5. Select a default resource group. This resource group is used to run tasks related to the data source, such as database SQL execution, offline full-database migration, and data preview.

6. Click Test Connection, or click OK to save the configuration and create the ClickHouse data source.

   Clicking Test Connection verifies the connection between Dataphin and the data source. If you click OK directly, the system automatically tests the connection for all selected clusters. The data source can be created even if the connection tests for all clusters fail.