This topic describes how to use Log Service SDK for Java to use the Dedicated SQL feature.

Prerequisites

Log Service SDK for Java V0.6.68 or later is installed. For more information, see Install Log Service SDK for Java.

Background information

Log Service provides the Dedicated SQL feature to enhance SQL analysis capabilities. You can use this feature to process hundreds of billions of rows of data. For more information, see Enable Dedicated SQL.
Log Service provides the executeLogStoreSql and executeProjectSql operations. You can call these operations to use the Dedicated SQL feature in an efficient manner.
  • executeLogStoreSql: uses the Dedicated SQL feature in a specified Logstore. This API operation supports the SQL-92 syntax. A query statement is in the Search statement|Analytic statement format and the analytic statement follows the standard SQL-92 syntax.
  • executeProjectSql: uses the Dedicated SQL feature in a specified project. This API operation supports the SQL-92 syntax. You must specify a filter condition and time range in the WHERE clause of an SQL statement.
Note If you want to filter data before you analyze the data, we recommend that you call the executeLogStoreSql operation and specify a query statement that is in the Search statement|Analytic statement format. This improves analysis efficiency.

Sample code

The following sample code shows how to use the Dedicated SQL feature. For more information, see Alibaba Cloud Log Service SDK for Java.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogContent;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.GetLogsResponse;

import java.util.Date;

public class SlsSample {

    public static void main(String args[]) throws LogException,
            InterruptedException {
        // The AccessKey pair that is used to access Log Service. For more information, see AccessKey pair. An Alibaba Cloud account has permissions to call all API operations. If you use the AccessKey pair of your Alibaba Cloud account, your resources are exposed to high security risks. We recommend that you log on as a RAM user that has permissions to call API operations or perform routine O&M tasks. 
        String accessId = "your_access_id";
        String accessKey = "your_access_key";
        // The name of the project. 
        String project = "aliyun-test-project";
        // The Log Service endpoint. For more information, see Endpoints. In this example, the Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint. 
        String host = "cn-hangzhou.log.aliyuncs.com"; 
        // The name of the Logstore. 
        String logStore = "aliyun-test-logstore";
        // The ID of the shard. 
        int shardId = 0;
        
        // Create a Log Service client. 
        Client client = new Client(host, accessId, accessKey);

        // Execute an SQL statement in the specified Logstore. 
        try {
            String sql = "* | select count(1)";
            int from = (int) (new Date().getTime() / 1000 - 600);
            int to = (int) (new Date().getTime() / 1000);
            GetLogsResponse logsResponse = client.executeLogstoreSql(project, logStore, 1627268185, 1627269085, "* | SELECT count(*)", true);
            System.out.println("Returned sql result count:" + logsResponse.GetCount());
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("time : " + item.mLogTime);
                for (LogContent content : item.mContents) {
                    System.out.println(content.mKey + ":" + content.mValue);
                }
            }
            // Print the statistics of the analysis result. 
            // The number of rows of log data that is processed. 
            System.out.println("proccesedRows:" + logsResponse.getProcessedRow());
            // The time that is required to execute the SQL statement. 
            System.out.println("elapsedMilli:" + logsResponse.getElapsedMilliSecond());
            // The CPU time that is consumed to execute the SQL statement after the Dedicated SQL feature is enabled. Unit: seconds. You are charged for the CPU time that is consumed when you use the Dedicated SQL feature to perform query and analysis operations. For more information, see Billable items. 
            System.out.println("cpuSec:" + logsResponse.getCpuSec());
            // The number of CPU cores that are used to execute the SQL statement after the Dedicated SQL feature is enabled. 
            System.out.println("cpuCores:" + logsResponse.getCpuCores());
            System.out.println("requestId:" + logsResponse.GetRequestId());

        } catch (LogException e) {
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }

        // Execute an SQL statement in the specified project. 
        try {
            int now = (int) (new Date().getTime() / 1000);
            String sql = "select count(1) as cnt from xxx where __time__ > " + now;
            GetLogsResponse logsResponse = client.executeProjectSql(project, "select avg(latency),max(latency) ,count(1) as c from sample-logstore where status>200 and __time__>=1500975424 and __time__ < 1501035044 GROUP BY method ORDER BY c", true);
            System.out.println("Returned sql result count:" + logsResponse.GetCount());
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                for (LogContent content : item.mContents) {
                    System.out.println(content.mKey + ":" + content.mValue);
                }
            }
            // Print the statistics of the analysis result. 
            // The number of rows of log data that is processed. 
            System.out.println("proccesedRows:" + logsResponse.getProcessedRow());
            // The time that is required to execute the SQL statement. 
            System.out.println("elapsedMilli:" + logsResponse.getElapsedMilliSecond());
            // The CPU time that is consumed to execute the SQL statement after the Dedicated SQL feature is enabled. Unit: seconds. You are charged for the CPU time that is consumed when you use the Dedicated SQL feature to perform query and analysis operations. For more information, see Billable items. 
            System.out.println("cpuSec:" + logsResponse.getCpuSec());
            // The number of CPU cores that are used to execute the SQL statement after the Dedicated SQL feature is enabled. 
            System.out.println("cpuCores:" + logsResponse.getCpuCores());
            System.out.println("requestId:" + logsResponse.GetRequestId());
        } catch (LogException e) {
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}
  • executeLogStoreSql operation
    You can call the executeLogStoreSql operation to use the Dedicated SQL feature. Requests must be in the GetLogsResponse logsResponse=client.executeLogstoreSql (project, logStore, from, to, query, powerSql) format. The following table describes the request parameters.
    Parameter Type Required Example Description
    project String Yes N/A The name of the project.

    When you create a Log Service client, you must specify a value for the project parameter. Therefore, you do not need to set the parameter again.

    logStore String Yes N/A The name of the Logstore.

    When you create a Log Service client, you must specify a value for the logStore parameter. Therefore, you do not need to set the parameter again.

    from Long Yes 1627268185 The start time of the time range that is specified in the request. The start time is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970.
    to Long Yes 1627269085 The end time of the time range that is specified in the request. The end time is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970.
    query String Yes "* | SELECT count(*)" The query statement. Format: Search statement|Analytic statement. For more information, see Syntax.

    By default, Log Service returns 100 rows of data. You can use a LIMIT clause to specify the number of rows of data to return. For more information, see LIMIT clause.

    powerSql Boolean No true Specifies whether to use the Dedicated SQL feature. For more information, see Enable Dedicated SQL.
    • true: The Dedicated SQL feature is used.
    • false: The Standard SQL feature is used. This is the default value.
  • executeProjectSql operation
    You can call the executeProjectSql operation to use the Dedicated SQL feature. Requests must be in the GetLogsResponse logsResponse=client.executeProjectSql (project, query, powerSql) format. The following table describes the request parameters.
    Parameter Type Required Example Description
    project String Yes N/A The name of the project.

    When you create a Log Service client, you must specify a value for the project parameter. Therefore, you do not need to set the parameter again.

    query String Yes "select avg(latency),max(latency) ,count(1) as c from sample-logstore where status>200 and __time__>=1500975424 and __time__ < 1501035044 GROUP BY method ORDER BY c" An SQL statement in which a filter condition and time range must be specified in the WHERE clause.

    By default, Log Service returns 100 rows of data. You can use a LIMIT clause to specify the number of rows of data to return. For more information, see LIMIT clause.

    powerSql Boolean No true Specifies whether to use the Dedicated SQL feature. For more information, see Enable Dedicated SQL.
    • true: The Dedicated SQL feature is used.
    • false: The Standard SQL feature is used. This is the default value.