All Products
Search
Document Center

Simple Log Service:SQL Enhancement

Last Updated:Apr 15, 2025

This topic describes the implementation and procedure of how to use SQL Enhancement. This way, you can use its core capabilities in an efficient manner.

Implementation

Background information

Simple Log Service separates storage from computing. The storage layer is abstracted as shards, which are used for data storage and management. The computing layer is abstracted as SQL, which provides query and analysis capabilities.

To improve the analysis performance of Simple Log Service, you must enhance the capabilities of the storage layer and the computing layer. The following sections provide common solutions and the comparison of the solutions.

image

Solution 1: Split a shard

You can manually split a shard to extend the capabilities of the storage layer. This way, the computing throughput is improved. For more information, see Split a shard.

Limits:

  • The method takes effect only on newly written data.

  • You are charged for active shards. For more information, see Billing description.

  • A query is subject to the maximum number of concurrent statements and the amount of data.

  • Manual operation is required for shard splitting or merging.

Solution 2: Divide a query into subqueries

You can divide a query into subqueries and perform secondary aggregation based on the results of the subqueries to obtain the final result.

Limits:

  • You must maintain the secondary aggregation logic and store the results of the subqueries. For more information, see Scheduled SQL.

  • The changes of the analysis dimension lead to the reconstruction of the aggregation logic.

  • Computing bottlenecks still exist when the amount of data is too large.

Solution 3: Enable SQL Enhancement

After you enable SQL Enhancement, Simple Log Service automatically scales storage and computing resources based on the amount of data to accelerate data analysis.

image

Comparison item

Solution 1

Solution 2

Solution 3

Resource isolation

Exclusive storage resources and shared computing resources.

Exclusive storage resources and shared computing resources.

Exclusive storage resources and elastic dedicated resource support.

Scaling

Storage layer scaling.

Computing layer scaling.

Auto scaling for the storage and computing layers.

O&M costs

Manual operation is required.

A query is divided into subqueries.

Fully automatic scheduling.

Query complexity

Native SQL support.

Custom development is required.

Native SQL support.

Prerequisites

Procedure

Simple Log Service allows you to enable SQL Enhancement based on the following methods:

  • Enabled once: After SQL Enhancement is enabled based on the enabled once method, only the query and analysis operations in the current Logstore can use SQL Enhancement.

  • Enabled by default: After SQL Enhancement is enabled based on the enabled by default method, all query and analysis operations, such as the alert- and dashboard-related operations, in the current project can use SQL Enhancement.

Console

Enabled once

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

    image

  3. On the Log Storage > Logstores tab, click the logstore that you want to manage.

    image

  4. Choose image > SQL Enhancement.

    image

Enabled by default

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

    image

  3. Click the 项目概览 icon.

    image.png

  4. Move your pointer over CUs of SQL-dedicated Instance, and then click Modify.

    SQL独享版

  5. In the Modify CUs of SQL-dedicated Instance panel, turn on Enable by Default, and then click OK.

API

Enabled once

  • GetLogs (the query results are not compressed and are directly returned)

    You need to only configure one of the following parameters:

    • powerSql: Valid values: true and false (default). The value true specifies that the current query uses SQL Enhancement. The value false specifies that the current query uses Standard SQL.

    • query: In this example, the * | select count(*) as pv query statement is executed. Add set session parallel_sql=true; to the analytic statement, indicating that SQL Enhancement is used. The query statement becomes * | set session parallel_sql=true; select count(*) as pv.

  • GetLogsV2 (the query results are compressed and then returned)

    You need to only configure one of the following parameters:

    • powerSql: Valid values: true and false (default). The value true specifies that the current query uses SQL Enhancement. The value false specifies that the current query uses Standard SQL.

    • query: In this example, the * | select count(*) as pv query statement is executed. Add set session parallel_sql=true; to the analytic statement, indicating that SQL Enhancement is used. The query statement becomes * | set session parallel_sql=true; select count(*) as pv.

Enabled by default

CreateSqlInstance: In this operation, set useAsDefault to true to automatically enable SQL Enhancement for your project.

SDK

In this example, Simple Log Service SDK for Java is used.

Prerequisites

Simple Log Service SDK for Java is installed. For more information, see Install Simple Log Service SDK for Java.

Enabled once

  • GetLogs() method

    You need to only configure one of the following parameters:

    • powerSql: Valid values: true and false (default). The value true specifies that the current query uses SQL Enhancement. The value false specifies that the current query uses Standard SQL.

    • query: In this example, the * | select count(*) as pv query statement is executed. Add set session parallel_sql=true; to the analytic statement, indicating that SQL Enhancement is used. The query statement becomes * | set session parallel_sql=true; select count(*) as pv.

    import com.aliyun.openservices.log.Client;
    import com.aliyun.openservices.log.exception.LogException;
    import com.aliyun.openservices.log.response.GetLogsResponse;
    
    import java.util.Date;
    
    public class CreateSqlInstance {
        public static void main(String[] args) throws LogException {
            // Configure environment variables. In this example, the AccessKey ID and AccessKey secret are obtained from environment variables.
            String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
            String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
            // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint.
            String host = "https://cn-beijing.log.aliyuncs.com";
            // Create a Simple Log Service client.
            Client client = new Client(host, accessId, accessKey);
            // The project name.
            String projectName = "aliyun-project-test";
            // The Logstore name.
            String logstore = "request_log";
            // Specify Whether to use SQL Enhancement. Default value: false.
            boolean useAsDefault = true;
            // Specify the query statement.
            String query = "* | select count(1)";
            // The maximum number of logs to return for the request. This parameter takes effect only when the query parameter is set to a search statement. Minimum value: 0. Maximum value: 100. Default value: 100.
            int line = 3;
            // The line from which the query starts. This parameter takes effect only when the query parameter is set to a search statement. Default value: 0.
            int offset = 0;
            // Specify whether to return logs in reverse chronological order of log timestamps. The log timestamps are accurate to the minute. Valid values:
            // true: Logs are returned in reverse chronological order of log timestamps.
            // false (default): Logs are returned in chronological order of log timestamps.
            boolean reverse = false;
            // Specify whether to use SQL Enhancement. Valid values:
            // true: uses SQL Enhancement.
            // false (default): uses Standard SQL.
            boolean powerSql = true;
            int from = (int) (new Date().getTime() / 1000 - 600);
            int to = (int) (new Date().getTime() / 1000);
            GetLogsResponse getLogsResponse = client.GetLogs(projectName, logstore, from, to, "", sql, line, offset, reverse, powerSql);
            System.out.println(getLogsResponse.getCpuSec());
        }
    }

Enabled by default

Set boolean useAsDefault to true to automatically enable SQL Enhancement for your project.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.CreateOrUpdateSqlInstanceRequest;

public class CreateSqlInstance {
    public static void main(String[] args) throws LogException {
        // Configure environment variables. In this example, the AccessKey ID and AccessKey secret are obtained from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // The project name.
        String projectName = "aliyun-test-project";
        // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint.
        String host = "https://cn-beijing.log.aliyuncs.com";
        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);
        int cu = 100;
        boolean useAsDefault = true;
        client.createSqlInstance(new CreateOrUpdateSqlInstanceRequest(projectName, cu, useAsDefault));
    }
}

FAQ

  • How do I obtain the amount of CPU time that I use?

    After you perform analysis and query operations, you can move the pointer over Analysis Results to view the amount of CPU time that you use. The following figure shows an example.计费

  • What are the fees of the Dedicated SQL feature when I execute a query statement once?

    The fees of the Dedicated SQL feature vary based on the amount of data on which you execute query statements. The following table provides examples.

    Query statement

    Amount of data (rows)

    Average cost per execution (USD)

    * | select avg(double_0) from stress_s1_mil1

    4 billion

    0.004435

    * | select avg(double_0), sum(double_0),), min(double_0), count(double_0) from stress_s1_mil1

    4 billion

    0.006504

    * | select avg(double_0), sum(double_1), max(double_2), min(double_3), count(double_4) from stress_s1_mil1

    4 billion

    0.013600

    * | select key_0 , avg(double_0) as pv from stress_s1_mil1 group by key_0 order by pv desc limit 1000

    4 billion

    0.011826

    * | select long_0, avg(double_0) as pv from stress_s1_mil1 group by long_0 order by pv desc limit 1000

    4 billion

    0.011087

    * | select long_0, long_1, avg(double_0) as pv from stress_s1_mil1 group by long_0,long_1 order by pv desc limit 1000

    0.3 billion

    0.010791

    * | select avg(double_0) from stress_s1_mil1 where key_0='key_987'

    4 billion

    0.00007