Create a Snowflake data source to enable Dataphin to read business data from Snowflake. This topic describes how to create a Snowflake data source.
Permissions
Only super administrators, data source administrators, domain architects, project administrators, and custom global roles with the Create Data Source permission can create data sources.
Procedure
On the Dataphin homepage, choose Management Center > Data Source Management in the top menu bar.
On the Data Source page, click + New Data Source.
In the Big Data section of the New Data Source page, select Snowflake.
If you have recently used Snowflake, you can also select it from the Recently Used section. Alternatively, you can enter `Snowflake` in the search box to find it quickly.
On the Create Snowflake Data Source page, configure the connection parameters.
Configure the basic information for the data source.
Parameter
Description
Datasource Name
Enter a name for the data source. The name must meet the following requirements:
Can contain Chinese characters, uppercase and lowercase letters, digits, underscores (_), and hyphens (-).
Cannot exceed 64 characters in length.
Datasource Code
After you configure the data source code, you can directly access Dataphin data source tables in Flink_SQL tasks or using the Dataphin Java Database Connectivity (JDBC) client. Use the format
DataSourceCode.TableNameorDataSourceCode.schema.TableNamefor quick access. To automatically switch data sources based on the task execution environment, use the variable format${DataSourceCode}.tableor${DataSourceCode}.schema.table. For more information, see Dataphin data source table development methods.ImportantThe data source code cannot be modified after it is configured.
You can preview data on the object details page in the asset directory and asset checklist only after the data source code is configured.
In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, SelectDB, and GaussDB data warehouse service (DWS) data sources are currently supported.
Version
Currently, only version 3.27.0 is supported.
Data Source Description
A brief description of the Snowflake data source. The description cannot exceed 128 characters.
Time Zone
Time format data in integration tasks is processed based on the selected time zone. The default time zone is Asia/Shanghai. Click Modify to select a different time zone. The options are:
GMT: GMT-12:00, GMT-11:00, GMT-10:00, GMT-09:30, GMT-09:00, GMT-08:00, GMT-07:00, GMT-06:00, GMT-05:00, GMT-04:00, GMT-03:00, GMT-03:00, GMT-02:30, GMT-02:00, GMT-01:00, GMT+00:00, GMT+01:00, GMT+02:00, GMT+03:00, GMT+03:30, GMT+04:00, GMT+04:30, GMT+05:00, GMT+05:30, GMT+05:45, GMT+06:00, GMT+06:30, GMT+07:00, GMT+08:00, GMT+08:45, GMT+09:00, GMT+09:30, GMT+10:00, GMT+10:30, GMT+11:00, GMT+12:00, GMT+12:45, GMT+13:00, GMT+14:00.
Daylight Saving Time: Africa/Cairo, America/Chicago, America/Denver, America/Los_Angeles, America/New_York, America/Sao_Paulo, Asia/Bangkok, Asia/Dubai, Asia/Kolkata, Asia/Shanghai, Asia/Tokyo, Atlantic/Azores, Australia/Sydney, Europe/Berlin, Europe/London, Europe/Moscow, Europe/Paris, Pacific/Auckland, Pacific/Honolulu.
Datasource Config
This setting depends on whether your business data source is separated into production and development environments:
If your business data source is separated, select Production + Development Data Source.
If your business data source is not separated, select Production Data Source.
Tag
You can add tags to classify data sources. For more information about how to create tags, see Manage data source tags.
Configure the connection parameters between the data source and Dataphin.
If you set Data Source Config to Production + Development Data Source, you must configure connection information for both. If you select Production Data Source, you only need to configure connection information for the production data source.
NoteTypically, the production and development data sources should be different to isolate the environments and reduce the impact of development activities on the production environment. However, Dataphin lets you configure them as the same data source using the same parameter values.
Parameter
Description
Server Address
Enter the server address in the format
{Account}.snowflakecomputing.com. Only one endpoint is supported. You cannot add more.Role
Enter the access role for Snowflake. Make sure the role has permission to use the warehouse.
Warehouse
Enter the Snowflake compute warehouse. This is the computing resource for data reads and writes. It must be in the running state.
Database
Enter the name of the target database. You must have access permissions for this database.
Schema
Enter the schema. The default value is PUBLIC.
Username, Password
Enter the username and password (or credentials) for authentication. To ensure tasks run correctly, make sure the user has the required data permissions.
Parameter Settings (Optional)
Click + Parameter Settings to add a row. Enter a Parameter Name and its corresponding Parameter Value. To delete a parameter, click the
icon in its row.Parameter names and values can contain uppercase and lowercase letters, digits, periods (.), underscores (_), and hyphens (-). They cannot exceed 256 characters in length.
Select the Default Resource Group. This resource group is used to run tasks related to this data source, such as database SQL tasks, full offline database migrations, and data previews.
Click Test Connection or OK to save the configuration and create the Snowflake data source.
Click Test Connection to verify that Dataphin can connect to the data source. If you click OK, the system automatically tests the connection for all selected clusters. The data source is created even if the connection tests fail.