Dataphin:Create an API data source

Permission requirements

Only users who have the Create Data Source permission point in a custom global role or users who have the super administrator, data source administrator, domain architect, or project administrator system role can create data sources.

Procedure

1. On the Dataphin homepage, click Management Hub > Datasource Management in the top navigation bar.

2. On the Datasource page, click +Create Data Source.

3. On the Create Data Source page, select API in the Semi-structured Storage section.

   If you have recently used API, you can also select API in the Recently Used section. You can also enter keywords in the search box to quickly search for API.

4. On the Create API Data Source page, configure the connection parameters.

   1. Configure the basic information of the data source.

      Parameter	Description
      Datasource Name	The name must meet the following requirements: It can contain only Chinese characters, letters, digits, underscores (_), and hyphens (-). It cannot exceed 64 characters in length.
      Datasource Code	After you configure the data source code, you can reference tables in the data source in a Flink_SQL node using the data source code.table name or data source code.schema.table name format. To automatically access the data source in the corresponding environment based on the current environment, use the variable format ${data source code}.table or ${data source code}.schema.table. For more information, see Development method for Dataphin data source tables. Important The data source code cannot be modified after it is configured successfully. After the data source code is configured successfully, you can preview data on the object details page in the asset directory and asset inventory. In Flink SQL, only MySQL, Hologres, MaxCompute, Oracle, StarRocks, Hive, and SelectDB data sources are currently supported.
      Data Source Description	A brief description of the data source. It cannot exceed 128 characters in length.
      Data Source Configuration	Select the data source that you want to configure: If the business data source distinguishes between production and development data sources, select Production + Development Data Source. If the business data source does not distinguish between production and development data sources, select Production Data Source.
      Tag	You can categorize data sources by adding tags. For information about how to create tags, see Manage data source tags.

   2. Configure the connection parameters between the data source and Dataphin.

      Note

      In most cases, the production data source and development data source must be configured as different data sources to achieve environment isolation between them and reduce the impact of the development data source on the production data source. However, Dataphin also supports configuring them as the same data source with identical parameter values.

      Parameter	Description
      URL Address	Enter the URL address of the API request.
      Authentication Type	Select the authentication method based on the API requirements. Basic Auth Username: Enter the username for the API. Password: Enter the password for the API. Alibaba Cloud AppKey Auth AppKey: Enter the AppKey for the API. AppSecret: Enter the AppSecret for the API. None: No authentication is required for the API. API Key Key: Enter the key for the API Key authentication method. Value: Enter the value for the API Key authentication method. Add To: Add the API key to one of the API request body parameters: Parameters, Headers, or Body. Bearer Token: Enter the token information. This information will be added to the API Headers in the format Authorization: Bearer <token> when making requests. OAuth2.0: Enter the Token Prefix, Access Token and configure the Access Token Acquisition Configuration below. Token Prefix (optional): Enter the token prefix. The default is Bearer. This field can be left empty. Access Token: Enter the JSON path of the Access Token in the response from the Access Token Acquisition Configuration. Multi-level paths are supported, such as data.access_token.
      Access Token Acquisition Configuration	Note This section is configurable only when the authentication method is set to OAuth2.0. Request Method: You can select either POST or GET. The default is GET. Token URL: Enter the request address for the token, in the format https://example.com/oauth/token. Client ID: Enter the client ID. Client Secret: Enter the client secret. Client Authentication: You can select either Send Basic Authentication Information In Request Header or Send Client Credentials In Request Body. The default is to send basic authentication information in the request header. Send Basic Authentication Information In Request Header: Sends basic authentication information directly through the Authorization header field in the HTTP request. The basic format for basic authentication is Authorization: Basic <credentials>, where <credentials> is the Base64-encoded username and password. Send Client Credentials In Request Body: Sends client authentication information in the request body, with key-value pairs in the format client_id, client_secret.
      Advanced Configuration	Note This section is configurable only when the authentication method is set to OAuth2.0. Request Parameters: You can enter multiple additional parameters required for requesting the token. This field is empty by default. When parameters entered here conflict with parameters automatically added by the authentication method, the parameters entered here take precedence. Parameter Name: Only letters, digits, underscores (_), and hyphens (-) are supported. The name cannot exceed 256 characters. Add To: You can select Parameter, Header, or Body. The default is Parameter. You can select Body only when the request method is POST. Connection Test: After you click Connection Test, the system automatically validates the Token URL, Client ID, Client Secret, and Client Authentication. After the connection test is complete, you can click Expand Query Results to view the formatted JSON.

   3. Advanced Settings

      Connection Retries: When the connection to the API fails, the system automatically retries the connection until the specified number of retries is reached. If the connection still fails after the specified number of retries, the connection is considered failed.

5. Click OK to complete the creation of the API data source.