Dataphin:Appendix: API call examples

SDK calls

Java SDK introduction

The Dataphin Data Service Java SDK provides you with the basic SDK and sample code for quickly calling Data Service APIs. This section introduces how to use the Dataphin service Java SDK.

The code file hierarchy of the Java SDK is as follows:

Java SDK/

• demo/

  • ClientDemo.java provides examples and methods for calling synchronous API operations.

  • AsyncClientDemo.java provides examples and methods for calling asynchronous API operations.

• lib/

  • dataphin-sdk-core-java-v5.5.0.jar is the dependency package for this SDK.

  • dataphin-sdk-core-java-v5.5.0-javadoc.jar is the Javadoc for the dependency package.

  • dataphin-sdk-core-java-v5.5.0-jar-with-dependencies.jar is the full dependency package.

• ApiDocument_v5.5.0.md is the API operation document.

• LICENSE is the license.

You can obtain the Java SDK as follows:

1. In the top menu bar on the Dataphin home page, choose Service > Application Management.

2. In the navigation pane on the left, click Call Instructions.

3. Click the API Call Instructions tab, select Java SDK Download in the upper-right corner of the page, and add the downloaded SDK JAR package to pom.xml.

Java SDK synchronous API call process

Step 1: Environment preparation

1. The Dataphin service Java SDK requires JDK 1.8 or later.

2. You need to prepare a pair of authentication keys for the SDK to generate authentication and signature information, namely AppKey and AppSecret.

   You can obtain the AppKey and AppSecret from the list on the Service > Application Management > My Applications page.

   Important

   AppKey and AppSecret are the keys used by the Dataphin Data Service to authenticate user requests. If these two configurations are stored on the client side, please encrypt them properly.

3. Add the following dependency to the pom.xml file. If an error occurs when you add dataphin-sdk-core-java, you must manually add the JAR file from the JAVA_SDK/lib/dataphin-sdk-core-java-v5.5.0.jar path. This JAR package is included in the Java SDK.

   Note

   dataphin-sdk-core-java is not available in the central Maven repository, so after you download the SDK, you need to upload it to your company's Maven repository or manually import it in IDEA or Eclipse.

   <dependency>
       <groupId>com.alibaba.dt</groupId>
       <artifactId>dataphin-sdk-core-java</artifactId>
       <version>v5.5.0</version>
   </dependency>

Step 2: Import the Java SDK API interface call class

1. You can download the corresponding API document from the Service > API Marketplace > API page.

2. Import `ClientDemo.java` and correct the import and package statements for the ClientDemo.java class.

   import com.alibaba.cloudapi.sdk.constant.SdkConstant;
   import com.alibaba.cloudapi.sdk.enums.Scheme;
   import com.alibaba.cloudapi.sdk.model.ApiCallback;
   import com.alibaba.cloudapi.sdk.model.ApiRequest;
   import com.alibaba.cloudapi.sdk.model.ApiResponse;
   import com.alibaba.dt.dataphin.client.ApiClient;
   import com.alibaba.dt.dataphin.client.ApiClientBuilderParams;
   import com.alibaba.dt.dataphin.client.sse.SseApiClient;
   import com.alibaba.dt.dataphin.schema.ManipulationParamRequest;
   import com.alibaba.dt.dataphin.schema.OrderBy;
   import com.alibaba.dt.dataphin.schema.QueryParamRequest;
   
   import java.util.*;
   import java.util.function.Consumer;
   
   /**
    * Synchronous API example
    * <p>
    * How to use:
    *  First, configure HOST, APP_KEY, APP_SECRET, and API_ID.
    *  Then, set the variables in getClient() and the parameter values in packQueryParamRequest() as needed.
    * Note:
    *  The parameters in getClient() may have different values for different APIs. You can modify getClient() to accept input parameters to adapt to different APIs.
    */
   public class ClientDemo {
   
       /**
        * The endpoint or IP address. Obtain this value from the network configurations of DataService Studio.
        */
       private static final String HOST = "xxx";
       /**
        * The AppKey of the application that you use to call this API.
        */
       private static final String APP_KEY = "xxx";
       /**
        * The AppSecret of the application that you use to call this API.
        */
       private static final String APP_SECRET = "xxx";
       /**
        * The ID of the API to call.
        */
       private static final String API_ID = "xxx";
   
       public static void main(String[] args) throws Exception {
           // Synchronously call a GET API.
           syncGet();
           // Synchronously call a LIST API.
           syncList();
           // Asynchronously call a GET API.
           asyncGet();
           // Asynchronously call a LIST API.
           asyncList();
           // Stream-call a GET API.
           fluxGet();
           // Paged query API that also obtains the total number of queries.
           syncListWithTotalNum();
   
           // Synchronously call a CREATE API.
           syncCreate();
           // Synchronously call an UPDATE API.
           syncUpdate();
           // Synchronously call a DELETE API.
           syncDelete();
       }
   
       private static void syncListWithTotalNum() {
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParamWithReturnNum();
   
           ApiResponse response = client.listSync(API_ID, queryParamRequest);
   
           String resultStr = getResultString(response);
   
           // contain totalNum in ResultBody
           System.out.println(resultStr);
       }
   
       /**
        * Synchronously call a GET API.
        */
       public static void syncGet() {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           ApiResponse response = client.getSync(API_ID, queryParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Synchronously call a LIST API.
        */
       public static void syncList() {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           ApiResponse response = client.listSync(API_ID, queryParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Synchronously call a CREATE API.
        */
       public static void syncCreate() {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
   
           // Modify the input parameters as needed.
           ManipulationParamRequest manipulationParamRequest = packDmlRequestParam();
   
           ApiResponse response = client.createSync(API_ID, manipulationParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Synchronously call an UPDATE API.
        */
       public static void syncUpdate() {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
   
           // Modify the input parameters as needed.
           ManipulationParamRequest manipulationParamRequest = packDmlRequestParam();
   
           ApiResponse response = client.updateSync(API_ID, manipulationParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Synchronously call a DELETE API.
        */
       public static void syncDelete() {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
   
           // Modify the input parameters as needed.
           ManipulationParamRequest manipulationParamRequest = packDmlRequestParam();
   
           ApiResponse response = client.deleteSync(API_ID, manipulationParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Asynchronously call a GET API.
        * You need to process the response in the callback method.
        */
       public static void asyncGet() throws Exception {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           client.getAsync(API_ID, queryParamRequest,
                   new ApiCallback() {
                       @Override
                       public void onFailure(ApiRequest request, Exception e) {
                           e.printStackTrace();
                       }
   
                       @Override
                       public void onResponse(ApiRequest request, ApiResponse response) {
                           try {
                               System.out.println(getResultString(response));
                           } catch (Exception e) {
                               e.printStackTrace();
                           }
                       }
                   });
           System.out.println("--- The main thread ends early. ---");
       }
   
       /**
        * Asynchronously call a LIST API.
        * You need to process the response in the callback method.
        */
       public static void asyncList() throws Exception {
           // Obtain the ApiClient object.
           ApiClient client = getClient();
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           client.listAsync(API_ID, queryParamRequest,
                   new ApiCallback() {
                       @Override
                       public void onFailure(ApiRequest request, Exception e) {
                           e.printStackTrace();
                       }
   
                       @Override
                       public void onResponse(ApiRequest request, ApiResponse response) {
                           try {
                               System.out.println(getResultString(response));
                           } catch (Exception e) {
                               e.printStackTrace();
                           }
                       }
                   });
           System.out.println("--- The main thread ends early. ---");
       }
   
       /**
        * Obtain the ApiClient object.
        */
       public static ApiClient getClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // The AppKey of the application.
           params.setAppKey(APP_KEY);
           // The AppSecret of the application.
           params.setAppSecret(APP_SECRET);
           // The endpoint or IP address.
           params.setHost(HOST);
           // The default protocol is HTTP. You can set the protocol to HTTPS based on the API.
           // Note: The built-in gateway of Dataphin does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The data environment for accessing the API. RELEASE specifies the production environment, and PRE specifies the development environment.
           params.setStage("RELEASE");
           // If you use a second-level domain name or an independent domain name for which the environment is not specified, you must set the env parameter.
           // Valid values: PROD and PRE. If you use the basic mode, set this parameter to PROD. If you use the Dev-Prod mode, set this parameter to PROD to query data in the production database or PRE to query data in the development database.
           // If you have specified an independent domain name to access the environment, the env parameter does not take effect.
           params.setEnv("PROD");
   
           ApiClient apiClient = new ApiClient(params);
           // Set the timeout period and polling interval for the Impala database type. This is optional. The default timeout period is 300s, and the default polling interval is 800 milliseconds.
           // This parameter is valid only for Impala. You must also set this parameter when you call non-Impala APIs. You can keep the default values.
           apiClient.setImpalaTimeoutAndInterval(300, 800);
           return apiClient;
       }
   
       public static SseApiClient getSseClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // The AppKey of the application.
           params.setAppKey(APP_KEY);
           // The AppSecret of the application.
           params.setAppSecret(APP_SECRET);
           // The endpoint or IP address.
           params.setHost(HOST);
           // The default protocol is HTTP. You can set the protocol to HTTPS based on the API.
           // Note: The built-in gateway of Dataphin does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The data environment for accessing the API. RELEASE specifies the production environment, and PRE specifies the development environment.
           params.setStage("RELEASE");
           // If you use a second-level domain name or an independent domain name for which the environment is not specified, you must set the env parameter.
           // Valid values: PROD and PRE. If you use the basic mode, set this parameter to PROD. If you use the Dev-Prod mode, set this parameter to PROD to query data in the production database or PRE to query data in the development database.
           // If you have specified an independent domain name to access the environment, the env parameter does not take effect.
           params.setEnv("PROD");
           return new SseApiClient(params);
   
       }
   
       /**
        * Call an API that contains a consumer.
        */
       public static void fluxGet() {
           Consumer<ApiResponse> consumer = r -> System.out.println(new String(r.getBody()));
           SseApiClient sseApiClient = getSseClient();
           // For short-lived applications, you can shut down the client's thread pool to release resources after the task is complete.
           sseApiClient.getFlux(API_ID, packRequestParam())
                   .doOnError(t -> System.out.println("error:" + t))
               // For temporary runs, close the connection to release resources upon completion or failure.
                   .doFinally(s -> sseApiClient.shutdown())
                   .subscribe(consumer);
       }
   
       private static QueryParamRequest packRequestParamWithReturnNum() {
           QueryParamRequest queryParamRequest = packRequestParam();
           queryParamRequest.setReturnTotalNum(true);
           return queryParamRequest;
       }
   
       /**
        * Package the request object.
        */
       private static QueryParamRequest packRequestParam() {
           QueryParamRequest queryParamRequest = new QueryParamRequest();
   
           /********* API-related business parameter settings begin *********/
           /*
            * The proxy account type. This parameter is optional.
            * Note: To use the proxy mode, the following conditions must be met:
            *  1. The row-level permissions feature is enabled.
            *  2. The application is granted the proxy permissions.
            *  3. The API is associated with row-level permissions.
            *
            * When you use the proxy mode for authentication, you must specify the account type of the proxy user.
            *  ACCOUNT_NAME: the username in Dataphin.
            *  USER_ID: the unique ID in Dataphin.
            *  SOURCE_USER_ID: the account ID in the source system.
            * You need to configure this parameter only when DelegationUid is set. If you do not configure this parameter, the default type is USER_ID.
            */
           // queryParamRequest.setAccountType("USER_ID");
   
           /*
            * The proxy account ID. This parameter is optional.
            * The user to be accessed by proxy. You must enter the corresponding account ID based on the selected AccountType.
            * If you set this parameter and the API is associated with a row-level permission rule, the proxy mode is used for authentication.
            */
           // queryParamRequest.setDelegationUid("abcd");
   
           /*
            * The list of request parameters. You can leave optional parameters unspecified. You must specify required parameters. Otherwise, an error is returned.
            * The key specifies the query field and the value specifies the value of the query field.
            * For example, id is the request field name and 1 is the value of id. You can set multiple query parameters.
            * Note: For an IN-type parameter, use a List to wrap the parameter value.
            */
           Map<String, Object> conditions = new HashMap<>();
           conditions.put("id", 1);
           conditions.put("age", Arrays.asList(10, 20, 30));
           queryParamRequest.setConditions(conditions);
   
           /*
            * The list of returned parameters. This parameter is optional.
            * For example, specify to return id and name.
            * If a returned parameter does not exist or you do not have permissions on it, an error is reported. If you do not pass this parameter, the parameters for which the application has permissions are returned.
            */
           queryParamRequest.setReturnFields(Arrays.asList("id", "name"));
   
           /*
            * The sorting field. This parameter is optional.
            * Note: If you use Oracle or SQL Server, you must use sorting for pagination.
            * Specify whether to sort the parameters in ascending or descending order. You can set multiple fields to be sorted in ascending or descending order.
            * For example, sort the returned result by id in ascending order.
            */
           List<OrderBy> orderList = new ArrayList<>();
           // orderList.add(new OrderBy("id1", OrderBy.Order.ASC));
           // orderList.add(new OrderBy("id2", OrderBy.Order.DESC));
           queryParamRequest.setOrderBys(orderList);
   
           /*
            * The pagination parameters. This parameter is optional and takes effect only for LIST-type APIs.
            */
           // The position from which the data is retrieved.
           queryParamRequest.setPageStart(0);
           // The number of data entries to retrieve on each page.
           queryParamRequest.setPageSize(10);
   
           /*
            * The API version number. This parameter is optional and can be set only for APIs in the development environment.
            */
           // queryParamRequest.setApiVersion("V1");
   
           /********* API-related business parameter settings end *********/
   
   
           /********* Feature parameter settings begin *********/
   
           /*
            * Specifies whether to use the model cache.
            * Enabling this feature reduces the parsing frequency for the same API in a service unit that has the same input and output parameters. This improves query efficiency.
            */
           queryParamRequest.setUseModelCache(false);
   
           /*
            * Specifies whether to use the result cache.
            * If you enable this feature, the query results of the same API with the same conditions and returned fields are cached.
            * This feature is suitable for queries on data that does not change. It reduces the query frequency of the same SQL statements and improves query efficiency.
            * The default cache duration is 30 minutes. After version 3.5.6, you can set the cache duration on the API development page.
            * Note: If the result cache is not enabled for the API, this parameter is invalid.
            */
           queryParamRequest.setUseResultCache(false);
   
           /*
            * Specifies whether to keep the case of fields.
            * If the fields are case-sensitive, we recommend that you set this parameter to true.
            * If this parameter is set to false, the field names in the result returned by a direct connection API are in uppercase by default.
            */
           queryParamRequest.setKeepColumnCase(true);
   
           /********* Feature parameter settings end *********/
   
           return queryParamRequest;
       }
   
       /**
        * Package the DML request object.
        */
       private static ManipulationParamRequest packDmlRequestParam() {
           ManipulationParamRequest manipulationParamRequest = new ManipulationParamRequest();
   
           /*
            * The list of request parameters. You can leave optional parameters unspecified. You must specify required parameters. Otherwise, an error is returned.
            * The key is the field name, and the value is the value of the field.
            * For example, id is the request field name and 1 is the value of id. You can set multiple parameters.
            * Note: For an IN-type parameter, use a List to wrap the parameter value.
            */
           Map<String, Object> conditions = new HashMap<>();
           conditions.put("id", 1);
           conditions.put("age", Arrays.asList(10, 20, 30));
           manipulationParamRequest.setConditions(conditions);
   
           /*
            * If the API processes data in batches, you need to put the input parameters into batchConditions.
            * Note: If the data volume is a single entry, do not put conditions into batchConditions to prevent serialization issues.
            */
           List<Map<String, Object>> batchConditions = new ArrayList<>();
           // batchConditions.add(new HashMap<>(conditions));
           manipulationParamRequest.setBatchConditions(batchConditions);
   
           /*
            * The API version number. This parameter is optional and can be set only for APIs in the development environment.
            */
           // manipulationParamRequest.setApiVersion("V1");
   
           return manipulationParamRequest;
       }
   
       /**
        * Get the result string.
        */
       private static String getResultString(ApiResponse response) {
           /*
            * Step 1: How to get detailed information from the HttpResponse header
            *         response.getHeaders().get(key)
            *         Description: The key can be "date\server\transfer-encoding\keep-alive\vary\connection\content-type\x-ca-request-id\x-ca-error-message\x-ca-error-code".
            *              If the API call fails, the headers will contain x-ca-error-message (error message description) and x-ca-error-code (gateway error code). If the call is successful, these headers will not be present.
            *              However, x-ca-request-id (the unique ID for each request) is always present.
            *         Recommendation: If the returned code is not 200, print x-ca-error-message and x-ca-error-code to troubleshoot the issue.
            *         Required: Print x-ca-request-id during use to facilitate log checking.
            *
            * Step 2: The detailed information of the response is printed below. In actual development, you can choose to output relevant information as described in Step 1.
            */
           // System.out.println("Detailed response information: "+ SdkConstant.CLOUDAPI_LF + JSON.toJSONString(response) + SdkConstant.CLOUDAPI_LF);
   
           StringBuilder result = new StringBuilder();
           result.append("Response from backend server").append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           result.append("ResultCode:").append(SdkConstant.CLOUDAPI_LF).append(response.getCode()).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           result.append("RequestId:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-request-id")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           if (200 != response.getCode()) {
               result.append("ErrorCode:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-error-code")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
               result.append("ErrorMessage:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-error-message")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           }
           result.append("ResultBody:").append(SdkConstant.CLOUDAPI_LF).append(new String(response.getBody(), SdkConstant.CLOUDAPI_ENCODING));
   
           /*
            * Check the result. If the status code is not 200, you can check the x-ca-error-message for troubleshooting.
            * We recommend that you save the x-ca-request-id parameter. If an exception occurs, provide this parameter for us to troubleshoot the issue.
            */
           return result.toString();
       }
   
   }

Step 3: Initialize the communication channel class

To call an API operation, you must first initialize the client. You can refer to the getClient method in ClientDemo.java and use the ApiClientBuilderParams class for initialization. You must replace the `Host`, `APP_Key`, and `APP_Secret` variables.

Step 4: API interface description

• API request methods

  • API request methods are categorized as query types (LIST and GET) or operation types (CREATE, UPDATE, and DELETE).

  • LIST supports paged queries, but GET does not. To set the parameters in QueryParamRequest, see the packRequestParam method in ClientDemo.java.

  • For operation-type API operations, see the packDmlRequestParam method in ClientDemo.java to set the parameters in ManipulationParamRequest.

• Calling API interfaces

  • The LIST and GET methods in the SDK are general methods for Dataphin Data Service. The SDK provides both synchronous and asynchronous calling methods. Before calling an API, you need to fill in the relevant request parameters according to the API you defined in Dataphin Data Service.

  • In this use case, conditions specifies the business request parameters for query API operations, and returnFields specifies the API return parameters. For more information about the parameters, navigate to the Service > Application Management > Authorized API Services page. In the Actions column for the target API operation, click Test to open the test page.

  • For API operations, business parameters are passed differently depending on the data volume configured for the API. For single-entry data, pass the input parameters in the conditions parameter. For batch data, pass the input parameters in the batchConditions parameter.

  • You can find the API_ID for the request parameter in the API column on the Service > Application Management > Authorized API Services page.

Java SDK asynchronous API call process

Step 1: Environment preparation

1. The Dataphin service Java SDK requires JDK 1.8 or later.

2. You need to prepare a pair of authentication keys for the SDK to generate authentication and signature information, namely AppKey and AppSecret.

   You can obtain the AppKey and AppSecret from the list on the Service > Application Management > My Applications page.

   Important

   AppKey and AppSecret are the keys used by the Dataphin service to authenticate user requests. If these two configurations are stored on the client side, please encrypt them properly.

3. Add the following dependency to the pom.xml file. If an error occurs when you add dataphin-sdk-core-java, you must manually add the JAR file from the JAVA_SDK/lib/dataphin-sdk-core-java-v5.5.0.jar path. This JAR package is located in the Java SDK.

   Note

   dataphin-sdk-core-java does not exist in the central Maven repository. Therefore, after you download the SDK, you must upload it to your company's Maven repository or manually import it in IDEA or Eclipse.

   <dependency>
       <groupId>com.alibaba.dt</groupId>
       <artifactId>dataphin-sdk-core-java</artifactId>
       <version>v5.5.0</version>
   </dependency>

Step 2: Import the Java SDK API interface call class

1. You can download the corresponding API document from the list on the Service > API Marketplace > API page.

2. Import AsyncClientDemo.java and correct the import and package statements for the AsyncClientDemo.java class.

   package com.alibaba.dt.dataphin;
   
   import com.alibaba.cloudapi.sdk.enums.Scheme;
   import com.alibaba.cloudapi.sdk.model.ApiRequest;
   import com.alibaba.dt.dataphin.client.ApiClientBuilderParams;
   import com.alibaba.dt.dataphin.client.DataphinDataServiceException;
   import com.alibaba.dt.dataphin.client.async.AsyncApiCallBack;
   import com.alibaba.dt.dataphin.client.async.AsyncApiClient;
   import com.alibaba.dt.dataphin.client.async.AsyncJobContext;
   import com.alibaba.dt.dataphin.schema.AsyncQueryResults;
   import com.alibaba.dt.dataphin.schema.OrderBy;
   import com.alibaba.dt.dataphin.schema.QueryParamRequest;
   import com.alibaba.fastjson.JSONObject;
   
   import java.util.ArrayList;
   import java.util.Arrays;
   import java.util.HashMap;
   import java.util.List;
   
   /**
    * Asynchronous API example
    * <p>
    * How to use:
    *  First, configure HOST, APP_KEY, APP_SECRET, and API_ID.
    *  Then, set the variables in getClient() and the parameter values in packQueryParamRequest() as needed.
    * Note:
    *  The parameters in getClient() may have different values for different APIs. You can modify getClient() to accept input parameters to adapt to different APIs.
    */
   public class AsyncClientDemo {
   
       /**
        * The endpoint or IP address. Obtain this value from the network configurations of DataService Studio.
        */
       private static final String HOST = "xxx";
       /**
        * The AppKey of the application that you use to call this API.
        */
       private static final String APP_KEY = "xxx";
       /**
        * The AppSecret of the application that you use to call this API.
        */
       private static final String APP_SECRET = "xxx";
       /**
        * The ID of the API to call.
        */
       private static final String API_ID = "xxx";
       /**
        * The number of entries to query in each batch. Default value: 1000.
        */
       private static final Integer FETCH_SIZE = 1000;
   
       public static void main(String[] args) throws Exception {
           //  Blocking call
           callAsyncApiBlock();
           //  Non-blocking call
           callAsyncApiNotBlock();
       }
   
       /**
        * Blocking call
        */
       @SuppressWarnings("all")
       private static void callAsyncApiBlock() {
   
           // Obtain the AsyncApiClient object.
           AsyncApiClient asyncApiClient = getClient();
   
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           try {
               AsyncQueryResults asyncQueryResults = asyncApiClient.listAsyncWaitFinish(API_ID, queryParamRequest);
   
               System.out.println(JSONObject.toJSONString(asyncQueryResults));
           } catch (Throwable e) {
               e.printStackTrace();
           }
   
           // Shut down the client to release resources.
           asyncApiClient.shutdown();
       }
   
       /**
        * Non-blocking call
        */
       @SuppressWarnings("all")
       private static void callAsyncApiNotBlock() {
   
           // Obtain the AsyncApiClient object.
           AsyncApiClient asyncApiClient = getClient();
   
           // Modify the return fields, query conditions, and other parameters as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           try {
               AsyncApiCallBack callback = new AsyncApiCallBack() {
                   @Override
                   public void onFailure(ApiRequest request, DataphinDataServiceException e) {
                       System.out.println(e.getMessage());
                   }
   
                   @Override
                   public void onResponse(ApiRequest request, AsyncQueryResults results) {
                       System.out.println(JSONObject.toJSONString(results));
                   }
               };
   
               AsyncJobContext context = asyncApiClient.listAsync(API_ID, queryParamRequest, callback);
   
               // If the request is successful, the jobId of the request is returned.
               System.out.printf("jobId: %s", context.getJobId());
   
               // Wait for the run to complete. In a real-world scenario, do not use the sleep method to wait.
               Thread.sleep(600000);
           } catch (Exception e) {
               e.printStackTrace();
           }
   
           // Shut down the client to release resources.
           asyncApiClient.shutdown();
       }
   
       /**
        * Obtain the AsyncApiClient object.
        */
       public static AsyncApiClient getClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // The AppKey of the application.
           params.setAppKey(APP_KEY);
           // The AppSecret of the application.
           params.setAppSecret(APP_SECRET);
           // The endpoint or IP address.
           params.setHost(HOST);
           // The default protocol is HTTP. You can set the protocol to HTTPS based on the API.
           // Note: The built-in gateway of Dataphin does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The number of entries to query in each batch. Default value: 1000.
           params.setFetchSize(FETCH_SIZE);
           // The stage environment variable. PRE specifies the API in the development environment, and RELEASE specifies the API in the production environment.
           params.setStage("RELEASE");
           // If you use a second-level domain name or an independent domain name for which the environment is not specified, you must set the env parameter.
           // Valid values: PROD and PRE. If you use the basic mode, set this parameter to PROD. If you use the Dev-Prod mode, set this parameter to PROD to query data in the production environment or PRE to query data in the development environment.
           // If you have specified an independent domain name to access an environment, the env parameter does not take effect.
           params.setEnv("PROD");
           return new AsyncApiClient(params);
       }
   
       /**
        * Package the request object.
        */
       private static QueryParamRequest packRequestParam() {
           QueryParamRequest queryParamRequest = new QueryParamRequest();
   
           /********* Configure API-related business parameters - start *********/
           /*
            * The type of the proxy account. This parameter is not required.
            * Note: To use the proxy mode, the following conditions must be met:
            *  1. The row-level permission feature is enabled.
            *  2. The application is granted the proxy permissions.
            *  3. The API is associated with a row-level permission rule.
            *
            * When you use the proxy mode for authentication, you must specify the account type of the proxy user.
            *  ACCOUNT_NAME: the username in Dataphin.
            *  USER_ID: the unique ID in Dataphin.
            *  SOURCE_USER_ID: the source system account ID.
            * This parameter is required only when DelegationUid is set. If you do not specify this parameter, the default value USER_ID is used.
            */
           queryParamRequest.setAccountType("USER_ID");
   
           /*
            * The ID of the proxy account. This parameter is not required.
            * The user to be accessed by proxy. You must enter the corresponding account ID based on the selected AccountType.
            * If you set this parameter and the API is associated with a row-level permission rule, the proxy mode is used for authentication.
            */
           queryParamRequest.setDelegationUid("abcd");
   
           /*
            * The list of request parameters. You can leave optional parameters unspecified. You must specify required parameters. Otherwise, an error is returned.
            * The key specifies the query field and the value specifies the value of the query field.
            * For example, id is the request field name and 1 is the value of id. You can specify multiple query parameters.
            * Note: If a parameter is of the IN type, use a List to wrap the parameter value, such as the age parameter.
            */
           HashMap<String, Object> conditions = new HashMap<>();
           conditions.put("id", 1);
           conditions.put("age", Arrays.asList(10,20,30));
           queryParamRequest.setConditions(conditions);
   
           /*
            * The list of returned parameters. This parameter is optional.
            * For example, you can specify to return the id and name parameters.
            * If a returned parameter does not exist or you do not have permissions on it, an error is reported. If you do not pass this parameter, the parameters for which the application has permissions are returned.
            */
           queryParamRequest.setReturnFields(Arrays.asList("id", "name"));
   
           /*
            * The sort field. This parameter is not required.
            * Note: If you use Oracle or SQL Server, you must use sorting for pagination.
            * You can specify one or more fields and the sort order (ascending or descending) for each field.
            * For example, you can sort the results by ID in ascending order.
            */
           List<OrderBy> orderList = new ArrayList<>();
           orderList.add(new OrderBy("id1", OrderBy.Order.ASC));
           orderList.add(new OrderBy("id2", OrderBy.Order.DESC));
           queryParamRequest.setOrderBys(orderList);
   
           /*
            * The version number of the API. This parameter is not required and can be set only in the development environment.
            */
           queryParamRequest.setApiVersion("V1");
   
           /********* Configure API-related business parameters - end *********/
   
   
           /********* Configure feature parameters - start *********/
   
           /*
            * Specifies whether to retain the case of fields. Set the value to true.
            */
           queryParamRequest.setKeepColumnCase(true);
   
           /********* Configure feature parameters - end *********/
   
           return queryParamRequest;
       }
   }

Step 3: Initialize the communication channel class

To call an API operation, you must first initialize the client. You can refer to the getClient method in AsyncClientDemo.java and use the ApiClientBuilderParams class for initialization. You must replace the Host, APP_Key, and APP_Secret variables.

Step 4: API interface description

• API request methods

  API request methods are categorized as LIST and GET. LIST supports paged queries, but GET does not. For more information, see the packRequestParam method in AsyncClientDemo.java to set the parameters in QueryParamRequest.

• Calling API interfaces

  • The LIST and GET methods in the SDK are general methods for Dataphin Data Service. The SDK provides both synchronous and asynchronous calling methods. Before calling an API, you need to fill in the relevant request parameters according to the API you defined in Dataphin Data Service.

  • In a use case, the conditions parameter specifies the business request parameters, and the returnFields parameter specifies the API response parameters. For more information about the parameters, go to the Service > Application Management > Authorized API Services page. In the Actions column for the target API operation, click Test to go to the test page.

  • To obtain the API_ID for the request parameter, go to the Service > Application Management > Authorized API Services page and find the API_ID in the API column.

Python SDK call process

Step 1: Environment preparation

1. Python SDK is applicable to Python 3.9 and later versions.

   You can obtain the Python SDK as follows:

   1. On the Dataphin home page, go to Service > Application Management in the top menu bar.

   2. In the navigation pane on the left, click Call Instructions.

   3. Click the API Call Examples tab, and click the Python Call Example Download button in the upper-right corner of the page to obtain the Python SDK core package.

2. Prepare a pair of authentication keys for the SDK to generate authentication and signature information, namely AppKey and AppSecret.

   Important

   AppKey and AppSecret are the keys used by the Dataphin service to authenticate user requests. If these two configurations are stored on the client side, please encrypt them properly.

3. Prepare a JSON file with the following format:

   {
     "host": "API gateway domain name",
     "port": 80,
     "impalaConfig": {
       "pollingTimeout": 300,
       "pollingInterval": 800
     },
     "applicationConfig": {
       "appKey": "Fill in sensitive data yourself",
       "appSecret": "Fill in sensitive data yourself"
     },
     "apiConfig": {
       "apiNo": 10008,
       "scheme": "HTTP",
       "stage": "RELEASE",
       "env": "PROD",
       "method": "LIST", // GET, LIST, CREATE, UPDATE, or DELETE. Case-sensitive.
       "queryParamRequest": {
         "conditions": {"id": "1"}, // The list of request parameters. Optional parameters can be omitted, but required parameters must be specified.
         "returnFields": ["id", "name", "age"], // The list of returned parameters. This parameter is optional. If a returned parameter does not exist or you do not have permissions on it, an error is reported. If you do not pass this parameter, the parameters for which the application has permissions are returned.
         "orderBys": [{"field": "id", "order": "ASC"}], // The sort fields. This parameter is optional.
         "useModelCache": "false", // Specifies whether to use the model cache. If you enable this parameter, the parsing frequency of an API in the same service unit is reduced when the input and output parameters are the same. This improves query efficiency.
         "useResultCache": "false", // Specifies whether to use the result cache. If you enable this parameter, the query results of an API are cached when the conditions and returned fields are the same.
         "apiVersion": "V1", // The API version number. This parameter is optional and can be set only in the development environment.
         "accountType": "USER_ID", // The proxy account type. This parameter is optional and is required when you use the proxy mode.
         "delegationUid": "abcd", // The proxy account ID. This parameter is optional and is required when you use the proxy mode.
         "returnTotalNum": "false" // Returns the total count. This may affect performance.
       },
       "manipulationParamRequest": {
         "conditions": {"id": "1"}, // Request parameters for an API with a "single" data volume. Optional parameters can be omitted, but required parameters must be specified.
         "batchConditions": [ // Request parameters for an API with a "batch" data volume. Optional parameters can be omitted, but required parameters must be specified.
           {"id": "1"},
           {"id": "2"}
         ]
       }
     }
   }

Step 2: Deploy Python SDK

1. Install the Python development tool PyCharm.

2. Open the Python project.

3. Configure the JSON file path in the startup parameters of the Demo class.

4. For specific calls, refer to demo.py.

Step 3: Call API

1. Configure call parameter information in the JSON file.

2. The Python SDK assembles the basic parameters for API calls by reading the JSON file.

3. Call the apiClient.callApi method. For more information, see the demo.py file.

   # -*- coding: utf-8 -*-
   import dataapi
   import sys
   
   with open(str(sys.argv[1]), encoding="utf-8") as f:
       json_obj = eval(f.read().replace('\n\u200b', ''))
   # This reads the context from the JSON file. If you do not need a JSON file, enter the corresponding values directly.
   
   # The address of the gateway.
   host = json_obj["host"]
   # The port of the gateway.
   port = json_obj["port"]
   # This parameter is optional and valid only for Impala-type APIs. The polling timeout period.
   pollingTimeout = json_obj["impalaConfig"]["pollingTimeout"]
   # This parameter is optional and valid only for Impala-type APIs. The polling interval.
   pollingInterval = json_obj["impalaConfig"]["pollingInterval"]
   # The information about the application that you use to call this API.
   appKey = json_obj["applicationConfig"]["appKey"]
   appSecret = json_obj["applicationConfig"]["appSecret"]
   # The information about the API.
   apiId = json_obj["apiConfig"]["apiNo"]
   # The method to call the API. Valid values: HTTP and HTTPS. Note: Private gateways support only HTTP. Case-sensitive.
   scheme = json_obj["apiConfig"]["scheme"]
   # The environment. RELEASE indicates the production environment.
   stage = json_obj["apiConfig"]["stage"]
   # Valid values: PROD and PRE. If you use the basic mode, set this parameter to PROD. If you use the Dev-Prod mode, PROD indicates that the production database is queried, and PRE indicates that the development database is queried. Case-sensitive.
   env = json_obj["apiConfig"]["env"]
   # Request parameters for a query API.
   queryParam = json_obj["apiConfig"]["queryParamRequest"]
   # Request parameters for a manipulation API.
   manipulationParam = json_obj["apiConfig"]["manipulationParamRequest"]
   # Specifies whether the API is of the GET, LIST, CREATE, UPDATE, or DELETE type. Case-sensitive.
   method = json_obj["apiConfig"]["method"]
   
   if (host == None or host == ""):
       raise Exception("The host information is missing.")
   
   if (appKey == None or appKey == ""):
       raise Exception("The appKey information is missing.")
   
   if (appSecret == None or appSecret == ""):
       raise Exception("The appSecret information is missing.")
   
   if (method == None or method == ""):
       raise Exception("The method information is missing.")
   
   if (apiId == None or apiId == ""):
       raise Exception("The apiNo information is missing.")
   
   # Configure Impala.
   impalaConfig = dataapi.ImpalaConfig(pollingTimeout=pollingTimeout, pollingInterval=pollingInterval)
   # Configure the application.
   appConfig = dataapi.AppConfig(appKey=appKey, appSecret=appSecret)
   apiConfig = dataapi.ApiConfig(apiId, scheme, stage, env, queryParam, method)
   myConfig = dataapi.MyConfig(host, port, impalaConfig, appConfig, apiConfig)
   
   apiClient = dataapi.ApiClient(myConfig)
   
   # Synchronous query API request.
   resp = apiClient.callApi(queryParam)
   print(resp)
   
   # Asynchronous query API request.
   asyncResp = apiClient.asyncCallApi(queryParam)
   print(asyncResp)
   
   # Flux API request.
   sseClient = dataapi.SseApiClient(myConfig)
   fluxResp = sseClient.fluxCallApi(queryParam)
   for item in fluxResp:
       print(item)
   
   # Synchronous manipulation API request.
   resp = apiClient.callApi(manipulationParam)
   print(resp)

Whitelist call method (supported only for private deployments)

By configuring a whitelist authentication-free access policy for APIs, you can simplify the API call process. The IP whitelist policy is an application-level policy, and currently only private cloud supports this call method.

Development process

Step 1: Configure the backend

You need to create an API, application, application whitelist, and bind the application and API in Data Service before you can access the API through the whitelist method.

• Create an API operation: On the Service > API Development > API Service page, select a method. For more information, see Create an API.

• Create an application and an application whitelist: Applications are used to call API operations in the production environment. You can also set whitelists for the applications. For information about how to create applications and whitelists, see Create and manage my applications.

• Application and API binding: You must request permissions from the application to which the API is attached to use the API. For more information, see Manage API permissions.

Step 2: Initiate a request

1. Request type: POST

   Only POST requests are supported. The request body must be a JSON string.

2. Set common input parameters

   Parameter name	Location	Required	Example	Description
   accept	Header	Yes	application/json; charset=utf-8	Response format.
   host	Header	Yes	gateway.aliyun.com	The API gateway domain name.
   x-ca-key	Header	Yes	Fill in sensitive data	appkey, the identity identifier for calling the API.
   x-ca-stage	Header	Yes	RELEASE	The environment identifier. RELEASE: accesses the production environment. PRE: accesses the development environment.
   Content-Type	Header	Yes	application/octet-stream; charset=utf-8	Request format.
   whitelist-flag	Header	Yes	1	The whitelist identifier. Set the value to 1.

3. Request format

   • The POST URL format is: http:// [host]/method/apiId?appKey=[appKey]&env=[env]

     • host: The API Gateway domain name. You can obtain this from the Service > Service Management > Network Configuration page.

     • method: The operation type of the API operation, which can be GET, LIST, CREATE, UPDATE, or DELETE. You can obtain the operation type from the Service > Application Management > Authorized API Services > Test page.

     • apiId: The unique identifier of the API operation. You can obtain this from the Service > Application Management > Authorized API Services > Test page.

     • appKey: The unique identifier of the application to which the API operation is bound. You can obtain this from the Service > Application Management > My Applications page.

     • env: Environment identifier, PROD indicates accessing the API in the production environment, PRE indicates accessing the API in the development environment.

     Example request URL: http://gateway.aliyun.com/list/12345?appKey=xxx&env=PROD.

4. Using PostMan call example

   1. Step 1: Request address

      Send a POST request to: http://[host]/method/apiId?appKey=[appKey]&env=[env].

   2. Step 2: Fill in Header

      Header example is as follows:

      accept:application/json;charset=utf-8
      x-ca-key:Your sensitive data
      host:Your API gateway domain name
      x-ca-stage:RELEASE
      Content-Type:application/octet-stream;charset=utf-8
      whitelist-flag:1

   3. Step 3: Fill in body

      The following code provides an example of the body:

      {
        "conditions": {"id": "1"}, // The list of request parameters. Optional parameters can be omitted, but required parameters must be specified.
        "batchConditions": [{"id": "1"}], // Request parameters for an operation-type API with a "batch" data volume. Optional parameters can be omitted, but required parameters must be specified.
        "returnFields": ["id", "name", "age"], // The list of returned parameters. This parameter is optional. If a returned parameter does not exist or you do not have permissions on it, an error is reported. If you do not pass this parameter, the parameters for which the application has permissions are returned.
        "orderBys": [{"field": "id", "order": "ASC"}], // The sort fields. This parameter is optional.
        "useModelCache": "false", // Specifies whether to use the model cache. If you enable this parameter, the parsing frequency of an API in the same service unit is reduced when the input and output parameters are the same. This improves query efficiency.
        "useResultCache": "false", // Specifies whether to use the result cache. If you enable this parameter, the query results of an API are cached when the conditions and returned fields are the same.
        "apiVersion": "V1", // The API version number. This parameter is optional and can be set only in the development environment.
        "accountType": "USER_ID", // The proxy account type. This parameter is optional and is required when you use the proxy mode.
        "delegationUid": "abcd" // The proxy account ID. This parameter is optional and is required when you use the proxy mode.
      }

Error codes

Client errors

Server-side errors (calling an API)

The following are API server errors.

Server-side errors (executing an SQL statement)

FAQ

Question 1: An API call returns a 404 error. Answer:

• If you have attached an independent domain, check whether the protocol is HTTP or HTTPS.

• Check that the operation type of the API you are calling is consistent with the API's defined type. The operation type is part of the URL, so ensure that you call the correct method.

• Check whether the API is published to the corresponding environment.

Question 2: An API call returns a 400 error. Answer:

• Check whether x-ca-timestamp is within the 15-minute validity period and whether x-ca-nonce has been reused within 15 minutes. For each API request, we recommend that you use the current time for x-ca-timestamp and generate a new x-ca-nonce. You can use a UUID to generate the `x-ca-nonce` value, which serves as a unique identifier and has no specific format requirements.

• Check whether there are spaces before or after the AppKey and AppSecret.

• Check that the API operation is authorized and that the AppKey and AppSecret of the authorized application match the APP_KEY and APP_SECRET parameters.

• Check whether the client's signature value contains extra spaces that are not present in the value sent to the server. Check the stringToSign value for spaces. For example, if the client signature Content-Type:application/octet-stream; charset=utf-8 contains a space but the value sent to the server is Content-Type:application/octet-stream;charset=utf-8, an Invalid Signature error is reported.

Question 3: An API call returns a 403 error.

Answer:

• Check whether the API is configured to use HTTP or HTTPS protocol, and set the corresponding parameter in your code.

• Check whether the AppKey and AppSecret are correct.

• Check that the operation type of the API you are calling is consistent with the API's defined type. The operation type is part of the URL, so ensure that you call the correct method.

Question 4: How do I use parameters with the IN operator?

Answer: When you use an SDK to make an API call, you must use a LIST to pass values for IN parameters. For example:

• If the parameter name is p1, the parameter type is String or Date, and the parameter values are a, b, c, you should set the parameter as follows:

Map<String, Object> conditions = Maps.newHashMap();
conditions.put("p1",Lists.newArrayList("a", "b", "c"));

• If the parameter is a numeric type with values 1, 2, and 3, set the parameter as follows:

Map<String, Object> conditions = Maps.newHashMap();
conditions.put("p1",Lists.newArrayList(1,2,3));

Question 5: When I use the SDK to retrieve paginated data, the total data count is correct, but some data is duplicated.

Reason: This occurs because the API code does not use a sort field to ensure a unique sorting result for each query, or because the database itself does not support sorting. This causes inconsistent sorting of query results each time data is retrieved with pagination, resulting in duplicate data or data loss.

Answer: Ensure that the sorting of the returned results is stable. If a primary key exists, add sorting by the primary key field. If no primary key exists, use multiple fields to form a composite primary key for sorting to ensure that the sorting result is stable for each query.

Question 6: When using the SDK to retrieve paginated data, how do I set parameters to return the total count?

Answer: In the packRequestParam method, set the parameters as follows:

// Specifies whether to return the total count (for APIs that support paged queries).
queryParamRequest.setReturnTotalNum(false);