Create a PolarDB data source - Dataphin - Alibaba Cloud Documentation Center

By creating a PolarDB data source, you can enable Dataphin to read business data from PolarDB or write data to PolarDB. This topic describes how to create a PolarDB data source.

Background information

PolarDB is a new generation of relational database independently developed by Alibaba Cloud. It is a cloud-hosted database product compatible with MySQL and PostgreSQL syntax. Under the storage-compute decoupled architecture, it leverages the advantages of software and hardware integration to provide users with database services featuring extreme elasticity, high performance, mass storage, and reliability. If you are using PolarDB, you need to first create a PolarDB data source before connecting to Dataphin for data development.

Policy description

Only custom global roles with the Create Data Source permission and the super administrator, data source administrator, domain architect, and project administrator roles can create data sources.

Procedure

In the top navigation bar of the Dataphin homepage, choose Management Center > Datasource Management.
On the Datasource page, click +Create Data Source.
On the Create Data Source page, in the Relational Database section, select PolarDB.
If you have recently used PolarDB, you can also select PolarDB in the Recently Used section. You can also enter PolarDB keywords in the search box to quickly search for it.

On the Create PolarDB Data Source page, configure the parameters for connecting to the data source.

Configure the basic information of the data source.

Parameter	Description
Datasource Name	Enter a name for the data source. The name must meet the following requirements: It can contain only Chinese characters, letters, digits, underscores (_), and hyphens (-). The name can be up to 64 characters in length.
Datasource Code	After configuring the data source code, you can directly access Dataphin data source tables in Flink_SQL tasks or using the Dataphin JDBC client through the format `data source code.table name` or `data source code.schema.table name` for quick consumption. To automatically switch data sources based on the task execution environment, please access using the variable format `${data source code}.table` or `${data source code}.schema.table`. For more information, see Dataphin data source table development method. Important The data source code cannot be modified after it is configured. You can preview data on the object details page in the asset directory and asset inventory only after the data source code is configured. In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, and SelectDB data sources are supported.
Data Source Description	A brief description of the PolarDB data source. It cannot exceed 128 characters.
Database Type	Based on the underlying engine type of PolarDB, select the corresponding database type: If the underlying engine of PolarDB is MySQL, select MYSQL. If the underlying engine of PolarDB is PostgreSQL, select POSTGRE_SQL.
Time Zone	Time-formatted data in integration nodes is processed based on the current time zone. The default time zone is Asia/Shanghai. Click Modify to select a destination time zone. The following options are available: GMT: GMT-12:00, GMT-11:00, GMT-10:00, GMT-09:30, GMT-09:00, GMT-08:00, GMT-07:00, GMT-06:00, GMT-05:00, GMT-04:00, GMT-03:00, GMT-03:00, GMT-02:30, GMT-02:00, GMT-01:00, GMT+00:00, GMT+01:00, GMT+02:00, GMT+03:00, GMT+03:30, GMT+04:00, GMT+04:30, GMT+05:00, GMT+05:30, GMT+05:45, GMT+06:00, GMT+06:30, GMT+07:00, GMT+08:00, GMT+08:45, GMT+09:00, GMT+09:30, GMT+10:00, GMT+10:30, GMT+11:00, GMT+12:00, GMT+12:45, GMT+13:00, GMT+14:00. Daylight Saving Time: Africa/Cairo, America/Chicago, America/Denver, America/Los_Angeles, America/New_York, America/Sao_Paulo, Asia/Bangkok, Asia/Dubai, Asia/Kolkata, Asia/Shanghai, Asia/Tokyo, Atlantic/Azores, Australia/Sydney, Europe/Berlin, Europe/London, Europe/Moscow, Europe/Paris, Pacific/Auckland, Pacific/Honolulu.
Data Source Configuration	Based on whether the business data source distinguishes between production data sources and development data sources: If the business data source distinguishes between production data sources and development data sources, select Production + Development Data Source. If the business data source does not distinguish between production data sources and development data sources, select Production Data Source.
Tag	You can categorize and tag data sources based on tags. For information about how to create tags, see Manage data source tags.

Configure the connection parameters between the data source and Dataphin.

If you select Production + Development Data Source for Data Source Configuration, you need to configure the connection information for both Production + Development Data Source. If you select Production Data Source for Data Source Configuration, you only need to configure the connection information for the Production Data Source.

Note

Typically, production data sources and development data sources should be configured as different data sources to achieve environment isolation between development data sources and production data sources, reducing the impact of development data sources on production data sources. However, Dataphin also supports configuring them as the same data source, meaning with identical parameter values.

Parameter	Description
JDBC URL	Configure the corresponding connection address based on the selected database type. MySQL: Format: `jdbc:mysql://host:port/dbname` PostgreSQL: Format: `jdbc:postgresql://host:port/dbname`
Schema	When the database type is selected as PostgreSQL in the previous step, you need to configure the Schema. Schema is the database name of the PostgreSQL instance.
Username, Password	The username and password used to log on to the PolarDB data source engine MySQL or PostgreSQL.

Configure advanced settings for the data source.

Parameter	Description
connectTimeout	The connectTimeout duration of the database (unit: milliseconds), default is 900000 milliseconds (15 minutes). Note If you have connectTimeout configuration in the JDBC URL, the connectTimeout will be the timeout period configured in the JDBC URL. For data sources created before Dataphin V3.11, the default connectTimeout is `-1`, indicating no timeout limit.
socketTimeout	The socketTimeout duration of the database (unit: milliseconds), default is 1800000 milliseconds (30 minutes). Note If you have socketTimeout configuration in the JDBC URL, the socketTimeout will be the timeout period configured in the JDBC URL. For data sources created before Dataphin V3.11, the default socketTimeout is `-1`, indicating no timeout limit.
Connection Retries	If the database connection times out, it will automatically retry connecting until the set number of retries is completed. If the maximum number of retries is reached and the connection is still unsuccessful, the connection fails. Note The default number of retries is 1 time, and you can configure a parameter between 0 and 10. The connection retry count will be applied by default to offline integration tasks and global quality (requires activation of the asset quality function module). In offline integration tasks, you can configure task-level retry counts separately.

Select Default Resource Group, which is used to run tasks related to the current data source, including database SQL, offline database migration, data preview, and more.
Perform a Test Connection or directly click OK to save and complete the creation of the PolarDB data source.
Click Test Connection, and the system will test whether the data source can connect normally with Dataphin. If you directly click OK, the system will automatically test the connection for all selected clusters, but even if all selected clusters fail to connect, the data source can still be created normally.