Dataphin:Configure API input components

Prerequisites

• An API data source is created. For more information, see Create an API data source.

• The account that is used to configure the properties of API input components has the read-through permission on the data source. If you do not have the permission, you must apply for the permission on the data source. For more information, see Apply for permissions on a data source.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Development > Data Integration.

2. In the top navigation bar of the Integration page, select a project (In Dev-Prod mode, you need to select an environment).

3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop to open the configuration page of the offline pipeline.

4. Click Component Library in the upper-right corner of the page to open the Component Library panel.

5. In the navigation pane on the left of the Component Library panel, select Inputs. Find the API component in the input component list on the right and drag it to the canvas.

6. Click the icon in the API input component card to open the API Input Configuration dialog box.

7. In the API Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	The name of the API input component. Dataphin automatically generates a step name. You can also modify the name based on your business scenario. The name must meet the following requirements: The name can contain only Chinese characters, letters, underscores (_), and digits. The name can be up to 64 characters in length.
   Datasource	The data source dropdown list displays all API data sources in the current Dataphin instance, including data sources on which you have the read-through permission and data sources on which you do not have the read-through permission. For data sources on which you do not have the read-through permission, you can click Request next to a data source to apply for the read-through permission on the data source. For more information, see Apply for permissions on a data source. If you do not have an API data source, click Create to create a data source. For more information, see Create an API data source.
   Request Method	POST and GET requests are supported. GET is selected by default. POST: You can configure Params, Headers, and Body. These parameters can be empty. GET: You can configure Params and Headers. These parameters can be empty.
   URL Path	Optional. Enter the path in the URL. The path parameter that you enter here is concatenated with the URL configured in the data source to form the request URL. For example, if the URL configured in the data source is https://www.aliyun.com and the URL path that you enter is /home, the request URL is https://www.aliyun.com/home.
   JSON Path To Read (Optional)	If you enter a JSON path, data is read based on the path, for example, data.result. If you do not enter a JSON path, the returned data is read.
   Number Of Requests	Single Request and Multiple Requests are supported. Single Request: Only one request is sent. Multiple Requests: Multiple requests are sent based on the specified loop parameter. When you select Multiple Requests, you need to configure the following parameters. Loop Mode: You can select Pagination Loop or Parameter Traversal Loop. Pagination loop Pagination Method: You can select Page Number/Offset or Cursor. Request Parameter: Configure the parameter for loop requests. You can select from Parameter or Body, or manually enter (supported only when the Body format is JSON). JSON path format such as a.b.c is supported, which is applicable to scenarios where the request body format is JSON. Parameter Value Source (configurable only when the pagination method is Cursor): The parameter value source must be a return parameter. You can enter a request update value, which supports JSON path. Start Value (Optional): When the pagination method is Page Number/Offset: Enter the start value for requests. The default value is 1. When the pagination method is Cursor: Enter the start value for requests. The default value is 0. Step Size (Optional, configurable only when the pagination method is Page Number/Offset): Enter the step size for loop requests. The default value is 1. Termination Condition: The loop stops when the termination condition is met. Left Condition: Configure the parameter source and parameter value. Parameter Source: You can select Request Parameter, Return Parameter, or Number Of Requests. The default value is Return Parameter. Parameter Value: Supports JSON path. When the parameter source is Request Parameter, you do not need to configure the operator and right condition. Operator: You can select Empty, Not Empty, =, ≠, ＞, ≥, ＜, or ≤. When you select Empty or Not Empty, you do not need to configure the right condition. Right Condition: Configure the parameter source and parameter value. Parameter Source: You can select Request Parameter, Return Parameter, or Constant. The default value is Constant. Parameter Value: Supports JSON path. Parameter traversal loop Request Parameter: Configure the parameter for loop requests. You can select from Parameter or Body, or manually enter (supported only when the Body format is JSON). JSON path format such as a.b.c is supported, which is applicable to scenarios where the request body format is JSON. Acquisition Method: Manual Entry: In the parameter value list area, fill in the loop list. You can use line breaks to separate values. API Acquisition: Currently only supports APIs without authentication. URL: Configure the request address for the parameter list. The request method can be POST or GET. The default value is GET. Request Parameters: Configure additional parameters for the request. The default value is empty. Click Create to add a row, and configure the parameter name, parameter value, and add to. Parameter Name and Parameter Value: Support all characters. Add To: You can select Params, Header, or Body. The default value is Params. Body is supported only when the request method is POST. Output Value: Fill in the output value. JSON path such as data.list is supported. Termination Condition: The loop ends when all selected parameter values have been traversed.
   Request Parameters	Request parameters include Params, Headers, Body (supported only when the request method is POST), and Signature. Params, headers, body Parameter Name: Chinese characters, letters, and special characters are supported. Parameter Type: You can select Constant, Timestamp, or Random Number. Parameter Value: If you select Constant as the parameter type, the parameter value can contain Chinese characters, letters, and special characters. If you select Timestamp as the parameter type, the parameter value can only be automatically generated. If you select Random Number as the parameter type, the parameter value can be automatically generated as a 16-bit, 32-bit, 64-bit, or 128-bit random number. Data Type: Only in Body, you can configure the data type. You can select String, Long, JSON, Double, Boolean, or Date. The default value is String. Actions: Create: Click +Create to add a row of parameters. Click the icon to save this parameter before you can create another parameter. Click the icon to cancel the parameter configuration. Edit: Click the icon in the Actions column of a configured parameter to edit the parameter configurations. Shift: Click the icon in the Actions column of a configured parameter to move the parameter to a list other than the current list. For example, if the current parameter is in the Params list, you can move it to the Header or Body list. If a parameter with the same name already exists in the target list, the parameter cannot be moved. If the target list is Body and the input method of Body is JSON, the parameter cannot be moved. Delete: Click the icon in the Actions column of a configured parameter to delete the parameter. If the parameter is referenced by a signature configuration, it cannot be deleted. The input format of Body can be JSON or Key-Value. JSON: You need to enter the corresponding JSON code in the JSON code editing area. Click the icon, and the system automatically formats (beautifies) the entered code. Key-Value: Click Create, enter the parameter name and parameter value, and select the parameter type and data type for the current parameter. Signature Signature is disabled by default. After you enable signature, you need to configure the rules for generating the signature. Signature Name: The key value of the encapsulated request parameter. Signature Location: You can select Params, Header, or Body (supported only when the request method is POST). Generation Function: Select MD5HEX, HMAC-MD5, SHA1HEX, HMAC-SHA256, HMAC-SHA1, SHA256, SHA256HEX, SHA512HEX, or HMAC_SHA512. Key: You can select a key from Administration > Data Security > Key Management or manually enter a key. If the current Dataphin instance has not purchased the Data Asset Security value-added module, only manual input is supported. If the current Dataphin instance has purchased the Data Asset Security value-added module, you can only select a key from Administration > Data Security > Key Management. If no key is available in the dropdown list, you can click Go To Register to go to Administration > Data Security > Key Management to register a key. Note This parameter is optional if you set Generation Function to MD5HEX, SHA1HEX, SHA256, SHA256HEX, or SHA512HEX. Concatenation Content: Select Parameter Value Only, Parameter Name + Parameter Value, or Custom. The default value is Parameter Name + Parameter Value. Parameter Value Only, Parameter Name + Parameter Value Referenced Parameters: Select multiple key-value (KV) parameters from the parameter lists. Concatenation Order: Select Dictionary Order or Selection Order. The default value is Dictionary Order. Connector (Optional): Configure a Parameter Separator or a Key-value Connector. Parameter Separator: Enter a custom parameter separator. The default value is &. Key-value Connector: Enter a custom key-value connector. The default value is =. The key-value connector cannot be configured if Concatenation Content is set to Parameter Value Only. Concatenation Example: Based on the preceding configurations, the concatenated string is displayed. If a timestamp or random number is used in the parameter, a temporarily generated value is displayed here. Custom Signature String: Enter a custom signature string. Enter the @ sign to trigger parameter suggestions. If you switch Concatenation Content from Parameter Value Only or Parameter Name + Parameter Value to Custom, and referenced parameters are included, the system concatenates them based on the concatenation order and connector, and then populates the signature string. The signature string can contain the key, parameters from Params, Header, or Body, and constants. If you select Key, #{secret} is added to the signature string. If you select All Params, All Header, or All Body, the system adds all parameters from the corresponding list to the signature string, with each parameter on a new line in K:V format. For example, if the parameter name is id, id:#{id} is added to the signature string. If the Body is in JSON format, the entire JSON object is passed to the system, and the #{body_json} placeholder is added to the signature string.
   Output Fields	The output fields are displayed. You can manually add output fields: Add output fields in batches. Format: Click Batch Add. You can configure output fields in batches in JSON format or TEXT format. JSON format: // Example: [{ "index": "0", "name": "user_id", "type": "String" }, { "index": "1", "name": "user_name", "type": "String" }] Note index indicates the column number of the specified object, name indicates the field name after import, and type indicates the field type after import. For example, "index":3,"name":"user_id","type":"String" indicates that the fourth column in the file is imported, the field name is user_id, and the field type is String. TEXT format: // Example: 0,user_id,String 1,user_name,String Row Delimiter and Column Delimiter: When the Format for batch adding is TEXT, you can configure the row delimiter and column delimiter. The row delimiter is used to separate the information of each field. The default value is the line feed \n. The supported values are \n ; . . The column delimiter is used to separate the field name and field type. The default value is a comma (,). Create a single output field. Click Create Output Field, and fill in Source Index and Field, and select Type as prompted on the page. Click the icon to save the current row, and click the icon to delete the current row. Manage output fields. For added fields, you can perform the following operations: Click the icon in the Operation column to edit existing fields. Click the icon in the Operation column to delete existing fields.

8. Click Confirm to complete the property configuration of the API input component.