Dataphin:Appendix: API call examples

SDK calls

Java SDK

The Dataphin data service Java SDK includes a base SDK and sample code for calling data service APIs.

The Java SDK has the following directory structure:

Java SDK/

• demo/

  • ClientDemo.java: Sample code and methods for synchronous APIs.

  • AsyncClientDemo.java: Sample code and methods for asynchronous APIs.

• lib/

  • dataphin-sdk-core-java-v6.3.0.jar: The core dependency package for the SDK.

  • dataphin-sdk-core-java-v6.3.0-javadoc.jar: The Javadoc for the dependency package.

  • dataphin-sdk-core-java-v6.3.0-jar-with-dependencies.jar: A standalone JAR package with all dependencies included.

• ApiDocument_v6.3.0.md: The API documentation.

• LICENSE: The license.

To obtain the Java SDK, follow these steps:

1. In the top navigation bar of the Dataphin homepage, choose Service > Application Management.

2. In the left navigation pane, click Usage Guide.

3. Click the API Calling Instructions tab. In the upper-right corner of the page, click Download SDK and select Download Java SDK. Add the downloaded SDK JAR package to your pom.xml file.

Java SDK API call flow

Step 1: Prepare your environment

1. The Dataphin Java SDK requires JDK 1.8 or later.

2. Prepare an access key pair (AppKey and AppSecret). The SDK uses this pair for authentication and to generate signatures.

   Obtain the AppKey and AppSecret on theServices >Application Management >My Applications page.

   Important

   The AppKey and AppSecret form a key pair that Dataphin uses to authenticate user requests. If this pair is stored on the client, ensure it is properly encrypted.

3. Add the following dependency to yourpom.xml file. If an error occurs when addingdataphin-sdk-core-java, manually add theJAVA_SDK/lib/dataphin-sdk-core-java-v6.3.0.jar file, which is included in theJava SDK.

   Note

   Thedataphin-sdk-core-java dependency is not available in the Maven Central Repository. After downloading the SDK, you must either upload it to your company's Maven repository or import it manually into your IDE (such as IntelliJ IDEA or Eclipse).

   <dependency>
       <groupId>com.alibaba.dt</groupId>
       <artifactId>dataphin-sdk-core-java</artifactId>
       <version>v6.3.0</version>
   </dependency>

Step 2: Import the API call class

1. Download the API documentation on theServices >API Marketplace >API page.

2. ImportClientDemo.java and correct theimport andpackage declarations in theClientDemo.java class.

   import com.alibaba.cloudapi.sdk.constant.SdkConstant;
   import com.alibaba.cloudapi.sdk.enums.Scheme;
   import com.alibaba.cloudapi.sdk.model.ApiCallback;
   import com.alibaba.cloudapi.sdk.model.ApiRequest;
   import com.alibaba.cloudapi.sdk.model.ApiResponse;
   import com.alibaba.dt.dataphin.client.ApiClient;
   import com.alibaba.dt.dataphin.client.ApiClientBuilderParams;
   import com.alibaba.dt.dataphin.client.sse.SseApiClient;
   import com.alibaba.dt.dataphin.schema.ManipulationParamRequest;
   import com.alibaba.dt.dataphin.schema.OrderBy;
   import com.alibaba.dt.dataphin.schema.QueryParamRequest;
   
   import java.util.ArrayList;
   import java.util.HashMap;
   import java.util.List;
   import java.util.Map;
   import java.util.function.Consumer;
   
   /**
    * API call examples.
    * <p>
    * Usage:
    *  1. Configure the HOST, APP_KEY, APP_SECRET, and API_ID constants.
    *  2. Set the variables in the getClient() method and parameter values in the packQueryParamRequest() method as needed.
    * Note:
    *  The parameters in getClient() may vary for different APIs. You can refactor getClient() to accept
    *  input parameters to accommodate different API requirements.
    */
   public class ClientDemo {
   
       /**
        * The host or IP address. Obtain this value from the data service's network configuration.
        */
       private static final String HOST = "xxx";
       /**
        * The AppKey of the calling application.
        */
       private static final String APP_KEY = "xxx";
       /**
        * The AppSecret of the calling application.
        */
       private static final String APP_SECRET = "xxx";
       /**
        * The ID of the API to call.
        */
       private static final String API_ID = "xxx";
   
       public static void main(String[] args) throws Exception {
           // Synchronous call for a GET API.
           syncGet();
           // Synchronous call for a LIST API.
           syncList();
           // Asynchronous call for a GET API.
           asyncGet();
           // Asynchronous call for a LIST API.
           asyncList();
           // Streaming call for a GET API.
           fluxGet();
           // Call a paginated query API that also retrieves the total number of records.
           syncListWithTotalNum();
   
       }
   
       private static void syncListWithTotalNum() {
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other settings as needed.
           QueryParamRequest queryParamRequest = packRequestParamWithReturnNum();
   
           ApiResponse response = client.listSync(API_ID, queryParamRequest);
   
           String resultStr = getResultString(response);
   
           // The ResultBody will contain `totalNum`.
           System.out.println(resultStr);
       }
   
       /**
        * Makes a synchronous call to a GET API.
        */
       public static void syncGet() {
           // Get an ApiClient object.
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other settings as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           ApiResponse response = client.getSync(API_ID, queryParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Makes a synchronous call to a LIST API.
        */
       public static void syncList() {
           // Get an ApiClient object.
           ApiClient client = getClient();
   
           // Modify the return fields, query conditions, and other settings as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           ApiResponse response = client.listSync(API_ID, queryParamRequest);
   
           System.out.println(getResultString(response));
       }
   
       /**
        * Makes an asynchronous call to a GET API.
        * The callback method must process the response.
        */
       public static void asyncGet() throws Exception {
           // Get an ApiClient object.
           ApiClient client = getClient();
           // Modify the return fields, query conditions, and other settings as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           client.getAsync(API_ID, queryParamRequest,
                   new ApiCallback() {
                       @Override
                       public void onFailure(ApiRequest request, Exception e) {
                           e.printStackTrace();
                       }
   
                       @Override
                       public void onResponse(ApiRequest request, ApiResponse response) {
                           try {
                               System.out.println(getResultString(response));
                           } catch (Exception e) {
                               e.printStackTrace();
                           }
                       }
                   });
           System.out.println("--- Main thread finished early. ---");
       }
   
       /**
        * Makes an asynchronous call to a LIST API.
        * The callback method must process the response.
        */
       public static void asyncList() throws Exception {
           // Get an ApiClient object.
           ApiClient client = getClient();
           // Modify the return fields, query conditions, and other settings as needed.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           client.listAsync(API_ID, queryParamRequest,
                   new ApiCallback() {
                       @Override
                       public void onFailure(ApiRequest request, Exception e) {
                           e.printStackTrace();
                       }
   
                       @Override
                       public void onResponse(ApiRequest request, ApiResponse response) {
                           try {
                               System.out.println(getResultString(response));
                           } catch (Exception e) {
                               e.printStackTrace();
                           }
                       }
                   });
           System.out.println("--- Main thread finished early. ---");
       }
   
       /**
        * Gets an ApiClient object.
        */
       public static ApiClient getClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // Application's AppKey.
           params.setAppKey(APP_KEY);
           // Application's AppSecret.
           params.setAppSecret(APP_SECRET);
           // Host or IP address.
           params.setHost(HOST);
           // The default protocol is HTTP. You can set it to HTTPS based on the API's requirements.
           // Note: The built-in Dataphin gateway does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The API's data environment. `RELEASE` for production environment, `PRE` for development environment.
           params.setStage("RELEASE");
           // If using a second-level domain or an independent domain without a specified environment, you must set `env`.
           // Supported values: `PROD` and `PRE`. For Basic mode, use `PROD`. For Dev-Prod mode, `PROD` queries the production database, and `PRE` queries the development database.
           // This parameter has no effect if you use an independent domain name that already specifies the environment.
           params.setEnv("PROD");
           // Connection timeout in milliseconds. Default: 10,000.
           params.setConnectionTimeout(60000L);
           // Read timeout in milliseconds. Default: 10,000.
           params.setReadTimeout(60000L);
   
           ApiClient apiClient = new ApiClient(params);
           // (Optional) Set the timeout and polling interval for Impala-type APIs. Default timeout: 300s. Default interval: 800ms.
           // This parameter only affects Impala APIs. For other APIs, you can keep the default values.
           apiClient.setImpalaTimeoutAndInterval(300, 800);
           return apiClient;
       }
   
       public static SseApiClient getSseClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // Application's AppKey.
           params.setAppKey(APP_KEY);
           // Application's AppSecret.
           params.setAppSecret(APP_SECRET);
           // Host or IP address.
           params.setHost(HOST);
           // The default protocol is HTTP. You can set it to HTTPS based on the API's requirements.
           // Note: The built-in Dataphin gateway does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The API's data environment. `RELEASE` for production environment, `PRE` for development environment.
           params.setStage("RELEASE");
           // If using a second-level domain or an independent domain without a specified environment, you must set `env`.
           // Supported values: `PROD` and `PRE`. For Basic mode, use `PROD`. For Dev-Prod mode, `PROD` queries the production database, and `PRE` queries the development database.
           // This parameter has no effect if you use an independent domain name that already specifies the environment.
           params.setEnv("PROD");
           // Connection timeout in milliseconds. Default: 10,000.
           params.setConnectionTimeout(60000L);
           // Read timeout in milliseconds. Default: 10,000.
           params.setReadTimeout(60000L);
   
           return new SseApiClient(params);
   
       }
   
       /**
        * Makes a streaming call to an API that includes a consumer.
        */
       public static void fluxGet() {
           Consumer<ApiResponse> consumer = r -> System.out.println(new String(r.getBody()));
           SseApiClient sseApiClient = getSseClient();
           // For short-lived applications, shut down the client's thread pool after the task is complete to release resources.
           sseApiClient.getFlux(API_ID, packRequestParam())
               .doOnError(t -> System.out.println("error:" + t))
               // For temporary runs, close the connection upon completion or failure to release resources.
               .doFinally(s -> sseApiClient.shutdown())
               .subscribe(consumer);
       }
   
       private static QueryParamRequest packRequestParamWithReturnNum() {
           QueryParamRequest queryParamRequest = packRequestParam();
           queryParamRequest.setReturnTotalNum(true);
           return queryParamRequest;
       }
   
       /**
        * Packages the request object.
        */
       private static QueryParamRequest packRequestParam() {
           QueryParamRequest queryParamRequest = new QueryParamRequest();
   
           /********* API Business Parameter Settings: Start *********/
           /*
            * (Optional) The type of the delegated account.
            * Note: To use delegation mode, all the following conditions must be met:
            *  1. Row-level security is enabled.
            *  2. The application is granted delegation permissions.
            *  3. The API is associated with a row-level security policy.
            *
            * When using delegation mode for authentication, specify the account type of the delegated user.
            *  - ACCOUNT_NAME: The Dataphin username.
            *  - USER_ID: The unique internal ID in Dataphin.
            *  - SOURCE_USER_ID: The account ID from the source system.
            * This parameter is required only when `DelegationUid` is set. The default type is `USER_ID`.
            */
           // queryParamRequest.setAccountType("USER_ID");
   
           /*
            * (Optional) The ID of the delegated account.
            * The delegated user. Pass the account ID corresponding to the selected `AccountType`.
            * If this parameter is set and the API is associated with a row-level security policy, delegation mode is used for authentication.
            */
           // queryParamRequest.setDelegationUid("abcd");
   
           /*
            * The list of request parameters. Optional parameters can be omitted. Required parameters must be provided, or an error will occur.
            * The `key` is the query field, and the `value` is its corresponding value.
            * For example, `id` is a request field name, and `1` is its value. You can set multiple query parameters.
            * Note: For `IN`-type parameters, wrap the values in a `List`.
            */
           Map<String, Object> conditions = new HashMap<>();
           // conditions.put("id", 1);
           // conditions.put("age", Arrays.asList(10, 20, 30));
           queryParamRequest.setConditions(conditions);
   
           /*
            * (Optional) The list of fields to return.
            * For example, specify `id` and `name`.
            * An error will occur if a specified field does not exist or the application lacks permission. If this list is not provided, the API returns all fields the application is authorized to access.
            */
           List<String> returnFields = new ArrayList<>();
           // returnFields.add("id");
           // returnFields.add("name");
           queryParamRequest.setReturnFields(returnFields);
   
           /*
            * (Optional) The fields to sort by.
            * Note: When using pagination with Oracle and SQL Server, you must also specify a sort order.
            * You can specify ascending or descending order for multiple fields.
            * For example, to sort the results by `id` in ascending order:
            */
           List<OrderBy> orderList = new ArrayList<>();
           // orderList.add(new OrderBy("id1", OrderBy.Order.ASC));
           // orderList.add(new OrderBy("id2", OrderBy.Order.DESC));
           queryParamRequest.setOrderBys(orderList);
   
           /*
            * (Optional) Pagination parameters. This applies only to LIST APIs.
            */
           // The starting index for the data retrieval.
           queryParamRequest.setPageStart(0);
           // The number of records to retrieve per page.
           queryParamRequest.setPageSize(10);
   
           /*
            * (Optional) The API version number. This can only be set for APIs in a development environment.
            */
           // queryParamRequest.setApiVersion("V1");
   
           /********* API Business Parameter Settings: End *********/
   
   
           /********* Feature Parameter Settings: Start *********/
   
           /*
            * Specifies whether to use the model cache.
            * Enabling this reduces parsing frequency for API calls with identical parameters within the same service unit, improving query efficiency.
            */
           queryParamRequest.setUseModelCache(false);
   
           /*
            * Specifies whether to use the result cache.
            * If enabled, the system caches query results for the same API with identical conditions and return fields.
            * This is suitable for queries on static data, as it reduces redundant SQL queries and improves efficiency.
            * The default cache duration is 30 minutes. Starting from v3.5.6, this can be configured on the API development page.
            * Note: This parameter is ineffective if result caching is not enabled for the API.
            */
           queryParamRequest.setUseResultCache(false);
   
           /*
            * Specifies whether to preserve the case of field names in the response.
            * Set this to `true` if your application is case-sensitive.
            * If `false`, field names in the response from a direct-connection API are uppercase by default.
            */
           queryParamRequest.setKeepColumnCase(true);
   
           /********* Feature Parameter Settings: End *********/
   
           return queryParamRequest;
       }
   
       /**
        * Packages the DML request object.
        */
       private static ManipulationParamRequest packDmlRequestParam() {
           ManipulationParamRequest manipulationParamRequest = new ManipulationParamRequest();
   
           /*
            * The list of request parameters. Optional parameters can be omitted. Required parameters must be provided, or an error will occur.
            * The `key` is the field name, and the `value` is its corresponding value.
            * You can set multiple parameters.
            * Note: For `IN`-type parameters, wrap the values in a `List`.
            */
           Map<String, Object> conditions = new HashMap<>();
           // conditions.put("id", 1);
           // conditions.put("age", Arrays.asList(10, 20, 30));
           manipulationParamRequest.setConditions(conditions);
   
           /*
            * For batch operations, add the input parameters to `batchConditions`.
            * Note: For single-record operations, do not place the `conditions` map into `batchConditions` to avoid serialization issues.
            */
           List<Map<String, Object>> batchConditions = new ArrayList<>();
           // batchConditions.add(new HashMap<>(conditions));
           manipulationParamRequest.setBatchConditions(batchConditions);
   
           /*
            * (Optional) The API version number. This can only be set for APIs in a development environment.
            */
           // manipulationParamRequest.setApiVersion("V1");
   
           return manipulationParamRequest;
       }
   
       /**
        * Gets the formatted result string.
        */
       private static String getResultString(ApiResponse response) {
           /*
            * Step 1: Getting details from the HttpResponse header.
            *         Use `response.getHeaders().get(key)`.
            *         - The `key` can be "date", "server", "transfer-encoding", "keep-alive", "vary", "connection", "content-type", "x-ca-request-id", "x-ca-error-message", or "x-ca-error-code".
            *         - If the API call fails, the header will contain `x-ca-error-message` (error description) and `x-ca-error-code` (gateway error code).
            *         - `x-ca-request-id` (the unique identifier for each request) is always included.
            *         - Recommended: If the response code is not 200, print `x-ca-error-message` and `x-ca-error-code` for troubleshooting.
            *         - Required: Always log `x-ca-request-id` to help trace requests.
            *
            * Step 2: The code below prints detailed response information. In a real application, you can customize the output based on Step 1.
            */
           // System.out.println("Detailed response information: " + SdkConstant.CLOUDAPI_LF + JSON.toJSONString(response) + SdkConstant.CLOUDAPI_LF);
   
           StringBuilder result = new StringBuilder();
           result.append("Response from backend server").append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           result.append("ResultCode:").append(SdkConstant.CLOUDAPI_LF).append(response.getCode()).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           result.append("RequestId:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-request-id")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           if (200 != response.getCode()) {
               result.append("ErrorCode:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-error-code")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
               result.append("ErrorMessage:").append(SdkConstant.CLOUDAPI_LF).append(response.getHeaders().get("x-ca-error-message")).append(SdkConstant.CLOUDAPI_LF).append(SdkConstant.CLOUDAPI_LF);
           }
           result.append("ResultBody:").append(SdkConstant.CLOUDAPI_LF).append(new String(response.getBody(), SdkConstant.CLOUDAPI_ENCODING));
   
           /*
            * Review the result. If the status code is not 200, check the `x-ca-error-message` to troubleshoot.
            * We recommend saving the `x-ca-request-id` parameter. If an exception occurs, provide this ID to technical support.
            */
           return result.toString();
       }
   
   }

Step 3: Initialize the client

To call an API, you must first initialize a client. See thegetClient method inClientDemo.java and use theApiClientBuilderParams class for initialization. You must replace the placeholder values for the HOST, APP_KEY, and APP_SECRET constants.

The following table describes the common parameters forApiClientBuilderParams.

Step 4: API method reference

• API request types

  • API requests are categorized as either query types (LIST, GET).

  • LIST requests support pagination, while GET requests do not. To configure query parameters, refer to thepackRequestParam method inClientDemo.java and set the properties of theQueryParamRequest object.

• Calling an API

  • The SDK provides both synchronous and asynchronous methods for calling Dataphin APIs. Before you call an API, you must provide the request parameters as defined in your API configuration within Dataphin.

  • In the code example,conditions holds the business request parameters for query APIs, andreturnFields specifies the response fields. To view the parameters for your API, go toServices >Application Management >Authorized API Services, find your API, and clickDebug in the Actions column.

  • To find the API_ID for a request, navigate toServices >Application Management >Authorized API Services and locate the ID in the API column.

• ApiClient call methods

  Note

  • Asynchronous calls use a non-blocking mechanism, allowing the main thread to continue its work without waiting. A callback handles the response, making this approach ideal for high-latency or long-running API calls.

  • Streaming calls useSseApiClient to implement Server-Sent Events (SSE) communication. Because SSE maintains a long-lived connection, you must call theshutdown() method when your task or short-lived application is complete to close the connection and release system resources.

Choose the ApiClient method that corresponds to your API's request type and desired call method:

Asynchronous API calls with the Java SDK

Step 1: Prepare the environment

1. The Dataphin Java SDK requires JDK 1.8 or later.

2. Prepare an access key, which consists of an AppKey and AppSecret. The SDK uses this key to generate authentication and signature information.

   Find your AppKey and AppSecret on the Services > Application Management > My Applications page.

   Important

   The AppKey and AppSecret form an access key used for authenticating requests to Dataphin services. Encrypt the access key if you store it on a client.

3. Add the following dependency to pom.xml. If adding dataphin-sdk-core-java causes an error, manually add the JAR file from JAVA_SDK/lib/dataphin-sdk-core-java-v6.3.0.jar. This file is included in the Java SDK package.

   Note

   dataphin-sdk-core-java is not available in the Maven Central Repository. Therefore, after you download the SDK, you must upload it to your company's Maven repository or manually import it into IDEA or Eclipse.

   <dependency>
       <groupId>com.alibaba.dt</groupId>
       <artifactId>dataphin-sdk-core-java</artifactId>
       <version>v6.3.0</version>
   </dependency>

Step 2: Set up the example code

1. Download the API documentation on the Services > API Marketplace > API page.

2. Import AsyncClientDemo.java and correct the import and package for the AsyncClientdemo.java class.

   package com.alibaba.dt.dataphin;
   
   import com.alibaba.cloudapi.sdk.enums.Scheme;
   import com.alibaba.cloudapi.sdk.model.ApiRequest;
   import com.alibaba.dt.dataphin.client.ApiClientBuilderParams;
   import com.alibaba.dt.dataphin.client.DataphinDataServiceException;
   import com.alibaba.dt.dataphin.client.async.AsyncApiCallBack;
   import com.alibaba.dt.dataphin.client.async.AsyncApiClient;
   import com.alibaba.dt.dataphin.client.async.AsyncJobContext;
   import com.alibaba.dt.dataphin.schema.AsyncQueryResults;
   import com.alibaba.dt.dataphin.schema.OrderBy;
   import com.alibaba.dt.dataphin.schema.QueryParamRequest;
   import com.alibaba.fastjson.JSONObject;
   
   import java.util.ArrayList;
   import java.util.Arrays;
   import java.util.HashMap;
   import java.util.List;
   
   /**
    * Sample code for asynchronous API calls.
    * <p>
    * Usage:
    *  Configure HOST, APP_KEY, APP_SECRET, and API_ID.
    *  Set the variables in getClient() and parameter values in packQueryParamRequest() as needed.
    * Note:
    *  The parameters in getClient() can vary by API. Modify getClient() to accept input parameters
    *  to support multiple APIs.
    */
   public class AsyncClientDemo {
   
       /**
        * The host or IP address, which you can find in the data service's network configuration.
        */
       private static final String HOST = "xxx";
       /**
        * The AppKey for the application calling this API.
        */
       private static final String APP_KEY = "xxx";
       /**
        * The AppSecret for the application calling this API.
        */
       private static final String APP_SECRET = "xxx";
       /**
        * The ID of the API to call.
        */
       private static final String API_ID = "xxx";
       /**
        * The number of entries to fetch per batch. Default: 1,000.
        */
       private static final Integer FETCH_SIZE = 1000;
   
       public static void main(String[] args) throws Exception {
           //  Blocking call example.
           callAsyncApiBlock();
           //  Non-blocking call example.
           callAsyncApiNotBlock();
       }
   
       /**
        * Blocking call.
        */
       @SuppressWarnings("all")
       private static void callAsyncApiBlock() {
   
           // Get an AsyncApiClient instance.
           AsyncApiClient asyncApiClient = getClient();
   
           // Customize return fields, query conditions, and other settings.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           try {
               AsyncQueryResults asyncQueryResults = asyncApiClient.listAsyncWaitFinish(API_ID, queryParamRequest);
   
               System.out.println(JSONObject.toJSONString(asyncQueryResults));
           } catch (Throwable e) {
               e.printStackTrace();
           }
   
           // Close the client and release resources.
           asyncApiClient.shutdown();
       }
   
       /**
        * Non-blocking call.
        */
       @SuppressWarnings("all")
       private static void callAsyncApiNotBlock() {
   
           // Get an AsyncApiClient instance.
           AsyncApiClient asyncApiClient = getClient();
   
           // Customize return fields, query conditions, and other settings.
           QueryParamRequest queryParamRequest = packRequestParam();
   
           try {
               AsyncApiCallBack callback = new AsyncApiCallBack() {
                   @Override
                   public void onFailure(ApiRequest request, DataphinDataServiceException e) {
                       System.out.println(e.getMessage());
                   }
   
                   @Override
                   public void onResponse(ApiRequest request, AsyncQueryResults results) {
                       System.out.println(JSONObject.toJSONString(results));
                   }
               };
   
               AsyncJobContext context = asyncApiClient.listAsync(API_ID, queryParamRequest, callback);
   
               // A successful call returns a jobId.
               System.out.printf("jobId: %s", context.getJobId());
   
               // Wait for the job to complete. Do not use Thread.sleep in production code.
               Thread.sleep(600000);
           } catch (Exception e) {
               e.printStackTrace();
           }
   
           // Close the client and release resources.
           asyncApiClient.shutdown();
       }
   
       /**
        * Gets an AsyncApiClient instance.
        */
       public static AsyncApiClient getClient() {
           ApiClientBuilderParams params = new ApiClientBuilderParams();
           // The application's AppKey.
           params.setAppKey(APP_KEY);
           // The application's AppSecret.
           params.setAppSecret(APP_SECRET);
           // The host or IP address.
           params.setHost(HOST);
           // The protocol is HTTP by default but can be set to HTTPS depending on the API.
           // Note: The Dataphin built-in gateway does not support HTTPS. Alibaba Cloud API Gateway supports HTTPS.
           params.setScheme(Scheme.HTTP);
           // The number of entries to fetch per batch. Default: 1,000.
           params.setFetchSize(FETCH_SIZE);
           // Stage environment: PRE for the development environment, RELEASE for the production environment.
           params.setStage("RELEASE");
           // If you use a custom domain where the environment is not specified, you must set the env parameter.
           // Valid values: PROD and PRE. In Basic mode, use PROD. In Dev-Prod mode, PROD queries production data and PRE queries development data.
           // This parameter has no effect if you have already specified an environment-specific domain.
           params.setEnv("PROD");
           return new AsyncApiClient(params);
       }
   
       /**
        * Builds the request object.
        */
       private static QueryParamRequest packRequestParam() {
           QueryParamRequest queryParamRequest = new QueryParamRequest();
   
           /********* Start of API-related business parameter settings *********/
           /*
            * (Optional) The type of the delegate account.
            * Note: Delegation mode requires the following:
            *  1. row-level security is enabled.
            *  2. The application has been granted delegation permissions.
            *  3. A row-level security policy is associated with the API.
            *
            * When using delegation mode for authentication, specify the delegated user's account type.
            *  ACCOUNT_NAME: The Dataphin username.
            *  USER_ID: The unique ID within Dataphin.
            *  SOURCE_USER_ID: The account ID from the source system.
            * Required only if DelegationUid is set. Defaults to USER_ID if unspecified.
            */
           // queryParamRequest.setAccountType("USER_ID");
   
           /*
            * (Optional) The ID of the delegate account.
            * The user on whose behalf the call is made. The ID must match the specified AccountType.
            * Setting this parameter enables delegation mode for authentication if the API has an associated row-level security policy.
            */
           // queryParamRequest.setDelegationUid("abcd");
   
           /*
            * Request parameters. Optional parameters can be omitted. Required parameters are mandatory.
            * The key is the query field name and the value is its corresponding value.
            * For example, 'id' is a request field with a value of 1. You can set multiple parameters.
            * Note: For IN-clause parameters, wrap the values in a List, as shown with the 'age' example.
            */
           HashMap<String, Object> conditions = new HashMap<>();
           conditions.put("id", 1);
           conditions.put("age", Arrays.asList(10,20,30));
           queryParamRequest.setConditions(conditions);
   
           /*
            * (Optional) The list of response fields.
            * For example, specify 'id' and 'name' to be returned.
            * An error occurs if a field does not exist or you lack permission. If omitted, all fields accessible to the application are returned.
            */
           List<String> returnFields = new ArrayList<>();
           // returnFields.add("id");
           // returnFields.add("name");
           queryParamRequest.setReturnFields(returnFields);
   
           /*
            * (Optional) The sorting fields.
            * Note: Oracle and SQL Server require sorting when using pagination.
            * Specify ascending or descending order. You can sort by multiple fields.
            * For example, sort by 'id' in ascending order.
            */
           List<OrderBy> orderList = new ArrayList<>();
           // orderList.add(new OrderBy("id1", OrderBy.Order.ASC));
           // orderList.add(new OrderBy("id2", OrderBy.Order.DESC));
           queryParamRequest.setOrderBys(orderList);
   
           /*
            * (Optional) The API version. Can only be set for APIs in the development environment.
            */
           // queryParamRequest.setApiVersion("V1");
   
           /********* End of API-related business parameter settings *********/
   
   
           /********* Start of feature parameter settings *********/
   
           /*
            * Specifies whether to preserve field name case. Must be true.
            */
           queryParamRequest.setKeepColumnCase(true);
   
           /********* End of feature parameter settings *********/
   
           return queryParamRequest;
       }
   }

Step 3: Initialize the client

To call an API, you must first initialize the client. You can refer to the getClient method in AsyncClientDemo.java and use the corresponding ApiClientBuilderParams class for initialization. You must replace the Host, APP_Key, and APP_Secret variables.

In addition to the common parameters such as appKey, appSecret, host, scheme, stage, and env listed in Step 3: Initialize the communication channel class of the synchronous API call process, AsyncApiClient also supports the following parameters that are exclusive to asynchronous calls:

Step 4: API call details

• API request methods

  API request methods include LIST and GET. The LIST method supports pagination, while the GET method does not. See the AsyncClientDemo.java file for the packRequestParam method, which you can use to set the parameters in QueryParamRequest.

• Call an API

  • The LIST and GET methods in the SDK are general-purpose methods for Dataphin data services. The SDK provides both synchronous and asynchronous call methods. Before calling an API, you must configure the request parameters according to the API's definition in the Dataphin data service.

  • In a debugging use case, conditions are the business request parameters, and returnFields are the API return parameters. For specific parameters, refer to the Service > Application Management > Authorized API Services page, and in the Actions column for the target API, click Debug to go to the debug page.

  • To find the API_ID for your request, navigate to the Services > Application Management > Authorized API Services page and find the ID in the API column.

• AsyncApiClient call methods

  Note

  After the call is complete, call asyncApiClient.shutdown() to close the client and release resources such as the thread pool. The example uses Thread.sleep to demonstrate the non-blocking effect. Do not use the sleep method to wait in real-world scenarios. We recommend that you handle the coordination between the callback and the main process in your business logic.

  AsyncApiClient provides two invocation methods: see the main method in AsyncClientDemo.java for examples:

  • Blocking call: When you call the listAsyncWaitFinish/getAsyncWaitFinish method, the main thread waits for the query to finish and directly returns an AsyncQueryResults object. This approach is suitable for scenarios where you need to synchronously obtain the complete results.

    Example: AsyncQueryResults results = asyncApiClient.listAsyncWaitFinish(API_ID, queryParamRequest);.

  • Non-blocking call: Call the listAsync/getAsync methods and pass an AsyncApiCallBack callback. The method immediately returns an AsyncJobContext. You can call context.getJobId() to obtain the job ID (jobId) of the query. After the query is complete, the result is returned through the onResponse method of the callback. If the call fails, exception information is returned through the onFailure method. This type of call is suitable for scenarios where you do not want to block the main thread.

    Example: AsyncJobContext context = asyncApiClient.listAsync(API_ID, queryParamRequest, callback);.

• List of AsyncApiClient methods

  Call type	Method	Applicable API type	Description
  Blocking	getAsyncWaitFinish(apiId, queryParamRequest)	GET	Wait for the query to complete and return the AsyncQueryResults object.
  Blocking	listAsyncWaitFinish(apiId, queryParamRequest)	LIST	Wait for the query to complete and return the AsyncQueryResults.
  Non-blocking	getAsync(apiId, queryParamRequest, callback)	GET	Processing the result through the AsyncApiCallBack callback returns an AsyncJobContext.
  Non-blocking	listAsync(apiId, queryParamRequest, callback)	LIST	The result is processed by the AsyncApiCallBack callback, and an AsyncJobContext is returned.
  Job Management	getJobStatus(jobId)	-	Queries the status of an asynchronous job.
  Job Management	getJobResult(jobId, fetchSize)	-	Fetches the query result of a job in batches.
  Job Management	cancelJob(jobId)	-	Cancels an asynchronous job.
  Job Management	closeJob(jobId)	-	Closes an asynchronous job and releases server-side resources.
  Job Management	getJobExecutionLog(jobId)	-	Obtains the execution log of an asynchronous job for troubleshooting.

Python SDK call process

Step 1: Prepare the environment

1. The Python SDK requires Python 3.9 or later.

   To get the Python SDK, follow these steps:

   1. In the top navigation bar of the Dataphin homepage, choose Services > Application Management.

   2. In the left-side navigation pane, click Call Instructions.

   3. Click the API Call Instructions tab. In the upper-right corner of the page, click SDK Download and select Download Python Call Example to download the Python SDK core package.

2. Prepare an authorization key pair, which consists of an AppKey and an AppSecret. The SDK uses this pair to generate authentication and signature information.

   Important

   The AppKey and AppSecret are credentials that Dataphin uses to authenticate user requests. If you store these credentials on a client, make sure to encrypt them securely.

3. Prepare a JSON file with the following format:

   {
     "host": "Your_API_gateway_domain",
     "port": 80,
     "impalaConfig": {
       "pollingTimeout": 300,
       "pollingInterval": 800
     },
     "applicationConfig": {
       "appKey": "Your_AppKey",
       "appSecret": "Your_AppSecret"
     },
     "apiConfig": {
       "apiNo": 10008,
       "scheme": "HTTP",
       "stage": "RELEASE",
       "env": "PROD",
       "method": "LIST", // The method can be GET, LIST, CREATE, UPDATE, or DELETE. This value is case-sensitive.
       "queryParamRequest": {
         "conditions": {"id": "1"}, // A list of request parameters. Required parameters must be included, while optional ones can be omitted.
         "returnFields": ["id", "name", "age"], // An optional list of return parameters. If a specified parameter does not exist or you lack permission to access it, an error occurs. If omitted, the API returns all parameters that your application has permission to access.
         "orderBys": [{"field": "id", "order": "ASC"}], // Optional sorting fields.
         "useModelCache": "false", // Enables the model cache. This reduces parsing frequency for API calls that have identical input and output parameters within the same service unit, improving query efficiency.
         "useResultCache": "false", // Enables the result cache. This caches query results for the same API call with identical conditions and return parameters.
         "keepColumnCase": "true", // Preserves the case of return parameter names.
         "apiVersion": "V1", // An optional API version. This can be set only in a development environment.
         "accountType": "USER_ID", // An optional delegated account type. This parameter is required when using delegation mode.
         "delegationUid": "abcd", // An optional delegated account ID. This parameter is required when using delegation mode.
         "returnTotalNum": "false" // Returns the total number of entries. Enabling this may affect performance.
       },
       "manipulationParamRequest": {
         "conditions": {"id": "1"}, // Request parameters for an API that processes a single entry. Required parameters must be included.
         "batchConditions": [ // Request parameters for an API that processes entries in a batch. Required parameters must be included.
           {"id": "1"},
           {"id": "2"}
         ]
       }
     }
   }

Step 2: Set up the Python SDK

1. Install a Python IDE, such as PyCharm.

2. Open your Python project.

3. Set the path to your JSON file as a startup argument for the demo class.

4. For a detailed call example, see the demo.py file.

Step 3: Call the API

1. Configure the call parameters in the JSON file.

2. The Python SDK reads the JSON file to assemble the basic parameters for the API call.

3. Call the apiClient.callApi method. For a detailed example, see the demo.py file.

   # -*- coding: utf-8 -*-
   import dataapi
   import sys
   
   # The following code reads the configuration from a JSON file.
   # Alternatively, you can assign values directly to the variables.
   with open(str(sys.argv[1]), encoding="utf-8") as f:
       json_obj = eval(f.read().replace('\n\u200b', ''))
   
   # The endpoint for the API gateway.
   host = json_obj["host"]
   # The port for the API gateway.
   port = json_obj["port"]
   # The polling timeout. This parameter applies only to Impala-type APIs.
   pollingTimeout = json_obj["impalaConfig"]["pollingTimeout"]
   # The polling interval. This parameter applies only to Impala-type APIs.
   pollingInterval = json_obj["impalaConfig"]["pollingInterval"]
   # The AppKey of the application that calls the API.
   appKey = json_obj["applicationConfig"]["appKey"]
   # The AppSecret of the application that calls the API.
   appSecret = json_obj["applicationConfig"]["appSecret"]
   # The ID of the API.
   apiId = json_obj["apiConfig"]["apiNo"]
   # The request protocol: HTTP or HTTPS. Note: The private gateway supports only HTTP. This value is case-sensitive.
   scheme = json_obj["apiConfig"]["scheme"]
   # The environment stage. Set this to 'RELEASE'.
   stage = json_obj["apiConfig"]["stage"]
   # The data environment. Valid values: PROD and PRE. In Basic mode, use PROD. 
   # In Dev-Prod mode, PROD queries the production database, and PRE queries the development environment. This value is case-sensitive.
   env = json_obj["apiConfig"]["env"]
   # The request parameters for a query API.
   queryParam = json_obj["apiConfig"]["queryParamRequest"]
   # The request parameters for a manipulation API.
   manipulationParam = json_obj["apiConfig"]["manipulationParamRequest"]
   # The request method: GET, LIST, CREATE, UPDATE, or DELETE. This value is case-sensitive.
   method = json_obj["apiConfig"]["method"]
   
   if (host is None or host == ""):
       raise Exception("host is missing")
   
   if (appKey is None or appKey == ""):
       raise Exception("appKey is missing")
   
   if (appSecret is None or appSecret == ""):
       raise Exception("appSecret is missing")
   
   if (method is None or method == ""):
       raise Exception("method is missing")
   
   if (apiId is None or apiId == ""):
       raise Exception("apiNo is missing")
   
   # Impala configuration
   impalaConfig = dataapi.ImpalaConfig(pollingTimeout=pollingTimeout, pollingInterval=pollingInterval)
   # Application configuration
   appConfig = dataapi.AppConfig(appKey=appKey, appSecret=appSecret)
   # API configuration
   apiConfig = dataapi.ApiConfig(apiId, scheme, stage, env, queryParam, method)
   myConfig = dataapi.MyConfig(host, port, impalaConfig, appConfig, apiConfig)
   
   apiClient = dataapi.ApiClient(myConfig)
   
   # Make a synchronous query API request.
   resp = apiClient.callApi(queryParam)
   print(resp)
   
   # Make an asynchronous query API request.
   asyncResp = apiClient.asyncCallApi(queryParam)
   print(asyncResp)
   
   # Make a streaming API request.
   sseClient = dataapi.SseApiClient(myConfig)
   fluxResp = sseClient.fluxCallApi(queryParam)
   for item in fluxResp:
       print(item)
   
   # Make a synchronous manipulation API request.
   resp = apiClient.callApi(manipulationParam)
   print(resp)

Whitelist API calls (private deployments only)

You can simplify API calls with a whitelist for password-free access. This application-level security policy is available only in private cloud environments.

Procedure

Step 1: Configure the backend

To call an API using a whitelist, you must first create an API and an application in Data Service, configure the application's whitelist, and authorize the application to call the API.

• Create an API: On the Service > API Development > API Service page, create an API. For more information, see Create an API.

• Create an application and a whitelist: Create an application to call APIs in a production environment and configure a whitelist for it. For more information about how to create an application and a whitelist, see Create and manage my applications.

• Authorize the application to call the API: Grant the application permission to call the API. For more information, see Manage API permissions.

Step 2: Send a request

1. Request method: POST

   Only POST requests are supported. The request body must be a JSON string.

2. Set common request parameters

   Parameter	Location	Required	Example	Description
   accept	Header	Yes	application/json; charset=utf-8	The response format.
   host	Header	Yes	gateway.aliyun.com	The API Gateway domain name.
   x-ca-key	Header	Yes	Your_appKey	The appKey that identifies the application calling the API.
   x-ca-stage	Header	Yes	RELEASE	The environment identifier. RELEASE specifies the production environment. PRE specifies the development environment.
   Content-Type	Header	Yes	application/octet-stream; charset=utf-8	The request format.
   whitelist-flag	Header	Yes	1	A flag that indicates a whitelist call. The value must be 1.

3. Request format

   • The POST URL format is: http://[host]/method/apiId?appKey=[appKey]&env=[env]

     • host: The API Gateway domain name. You can find this on the Service > Service Management > Network Configuration page.

     • method: The API request method: GET/LIST. You can find this on the Service > Application Management> Authorized API Services > Debug page.

     • apiId: The unique ID of the API. You can find this on the Service > Application Management> Authorized API Services > Debug page.

     • appKey: The unique identifier of the application that is bound to the API. You can find this on the Service > Application Management > My Applications page.

     • env: The environment identifier. Use PROD for the production environment and PRE for the development environment.

     Example request URL: http://gateway.aliyun.com/list/12345?appKey=xxx&env=PROD.

4. Example Postman call

   1. Step 1: Request URL

      The POST request URL is: http://[host]/method/apiId?appKey=[appKey]&env=[env].

   2. Step 2: Set the headers

      The following code block shows an example of headers:

      accept:application/json;charset=utf-8
      x-ca-key:Your_appKey
      host:Your_API_Gateway_domain
      x-ca-stage:RELEASE
      Content-Type:application/octet-stream;charset=utf-8
      whitelist-flag:1

   3. Step 3: Set the request body

      The following code block shows an example of a request body:

      {
        "conditions": {"id": "1"}, // Request parameter list. Pass optional parameters as needed. Required parameters must be provided.
        "batchConditions": [{"id": "1"}], // Request parameters for operation APIs with batch input. Pass optional parameters as needed. Required parameters must be provided.
        "returnFields": ["id", "name", "age"], // An optional list of return parameters. If a specified parameter does not exist or you lack permission to access it, an error occurs. If omitted, the API returns all parameters that you have permission to access.
        "orderBys": [{"field": "id", "order": "ASC"}], // Optional sorting fields.
        "useModelCache": "false", // Specifies whether to use the model cache. This feature improves query efficiency by reducing parsing frequency for API calls with identical parameters within the same service unit.
        "useResultCache": "false", // Specifies whether to use the result cache, which caches query results for identical API calls.
        "apiVersion": "V1", // Optional API version. Can be set only in a development environment.
        "accountType": "USER_ID", // Optional delegated account type. Required only for delegation mode.
        "delegationUid": "abcd" // Optional delegated account ID. Required only for delegation mode.
      }

Error codes

Client errors

Server-side errors (API calls)

The following errors are returned by the API server.

Server-side errors (SQL execution)

FAQ

Question 1: An API call returns a 404 error. Answer:

• If you are using a custom domain name, verify the request protocol (HTTP or HTTPS).

• The operation type is part of the URL. Ensure the method you call matches the API's defined operation type.

• Ensure the API is published to the correct environment.

Question 2: An API call returns a 400 error. Answer:

• Check that the x-ca-timestamp is within its 15-minute validity period and that the x-ca-nonce is not reused within 15 minutes. For each API request, set x-ca-timestamp to the current time and generate a new x-ca-nonce. Use a UUID to generate the nonce; it is a unique identifier with no specific format requirements.

• Verify that the AppKey and AppSecret do not have any leading or trailing spaces.

• Verify that the API is authorized for your application and that the AppKey and AppSecret in your request match those of the authorized application.

• Check for extra spaces in the signature value on the client side. An Invalid Signature error occurs if the stringToSign value on the client differs from what the server expects. For instance, if the client-side signature for the Content-Type header is Content-Type:application/octet-stream; charset=utf-8 (with a space) but the header sent to the server is Content-Type:application/octet-stream;charset=utf-8 (without a space), the signature validation will fail.

Question 3: An API call returns a 403 error.

Answer:

• Check whether the API is configured to use the HTTP protocol or the HTTPS protocol, and set the corresponding parameter in your code.

• Verify that the AppKey and AppSecret are correct.

• The operation type is part of the URL. Ensure the method you call matches the API's defined operation type.

Question 4: How do I use a parameter with the IN operator?

Answer: When calling an API with the SDK, pass the values for a parameter that uses the IN operator as a list. For example:

• Assume the parameter is named p1, its type is String or Date, and its values are 'a', 'b', and 'c'. Set the parameter as follows:

Map<String, Object> conditions = Maps.newHashMap();
conditions.put("p1",Lists.newArrayList("a", "b", "c"));

• If the parameter type is numeric and its values are 1, 2, and 3, set the parameter as follows:

Map<String, Object> conditions = Maps.newHashMap();
conditions.put("p1",Lists.newArrayList(1,2,3));

Question 5: When using the SDK to retrieve paginated data, the total record count is correct, but duplicate records are returned.

Cause: This issue occurs because the sort order of the query result is not stable. This can happen if the API code does not use a sort field to guarantee a unique order or if the underlying database does not support sorting. As a result, the order can change between paginated requests, leading to duplicate or missing data.

Answer: Ensure that the sort order of the results is stable. If a primary key exists, add it to the sort fields. If there is no primary key, use multiple fields to create a composite primary key for sorting. This guarantees a stable sort order for every request.

Question 6: When using the SDK to fetch data with pagination, how do I configure a parameter to return the total count?

Answer: In the packRequestParam method, set the parameter as follows:

// Specifies whether to return the total number of entries (for APIs that support pagination).
queryParamRequest.setReturnTotalNum(false);