All Products
Search
Document Center

Simple Log Service:Enable Dedicated SQL

Last Updated:Mar 13, 2024

If you use the Standard SQL feature to analyze a large amount of data, the logs within the specified time range cannot be fully scanned. In this case, the returned analysis results may not contain all matched data. You can increase the number of shards to improve data read and write capabilities. However, this method takes effect only for incremental data. You can enable the Dedicated SQL feature to increase computing resources and the amount of data that can be analyzed in a single query request. This topic describes how to enable Dedicated SQL.

Prerequisites

Background information

Data in Simple Log Service is stored in shards. For more information, see Shard. Simple Log Service limits the amount of data that can be scanned in each shard to manage the data read and write capabilities of Logstores, Eventstores, and Metricstores. If you use the Standard SQL feature to analyze a large amount of data, Simple Log Service can return only some matched results. You can increase the number of shards to improve data read and write capabilities. However, this method takes effect only for incremental data. The number of consumers also increases. For more information, see Overview of real-time consumption. The Dedicated SQL feature provides more powerful SQL analysis capabilities. You can use this feature in the following scenarios:

  • You need to analyze data with high performance. For example, you need to analyze data in real time.

  • You need to analyze data that is generated over a long period of time. For example, you need to analyze data that is generated over a month.

  • You need to analyze a large amount of data. For example, you need to analyze hundreds of billions of rows of data.

  • You need to analyze data by using more than 15 concurrent SQL statements and display the analysis results based on multiple metrics from multiple dimensions.

For information about the comparison between the Dedicated SQL and Standard SQL, see Log analysis overview.

Billing

  • Pay-as-you-go: Daily fee of Dedicated SQL = CPU time that is calculated by hour × Price per hour.

  • Resource plan: Prepaid resource plans are used to offset the fees of billable items based on the CUs of the billable items.

Procedure

Simple Log Service supports the following methods to enable the Dedicated SQL feature:

  • Enable once: When you execute a query statement in a Logstore, enable Dedicated SQL. The Dedicated SQL feature takes effect only for the query statements that you execute in the current Logstore.

  • Enable permanently: If you turn on Enable by Default, the Dedicated SQL feature is enabled for the current project and takes effect for all query statements that you execute in the project, including the query statements for alerts and dashboards.

Enable once

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the Logstore that you want to manage.

  4. Click the SQL独享版 icon.

    image.png

Enable permanently

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. Click the 项目概览 icon.

    image.png

  4. Move the pointer over the CUs of SQL-dedicated Instance parameter and click Modify.

    SQL独享版

  5. In the Modify CUs of SQL-dedicated Instance panel, turn on Enable by Default and click OK.

SDK examples

FAQ

  • How do I enable the Dedicated SQL feature by calling an API operation?

    You can enable the Dedicated SQL feature by calling the GetLogs operation. When you call this operation, you can use the powerSql or query parameter to specify whether to enable the Dedicated SQL feature. For more information, see GetLogs.

  • How do I obtain the amount of CPU time that I use?

    After you perform analysis and query operations, you can move the pointer over Analysis Results to view the amount of CPU time that you use. The following figure shows an example.计费

  • Can I control the cost of the Dedicated SQL feature?

    Yes, you can modify the number of compute units (CUs) to control the cost of the Dedicated SQL feature. To modify the number of CUs, go to the Project Overview page of your project and change the value of the CUs of SQL-dedicated Instance parameter.

    Note

    When you use the Dedicated SQL feature, CUs are used in parallel.

    核数

  • What are the fees for the Dedicated SQL feature when I execute a query statement once?

    The fees for the Dedicated SQL feature vary based on the amount of data on which you execute query statements. The following table provides examples.

    Query statement

    Amount of data (rows)

    Average cost per execution (USD)

    * | select avg(double_0) from stress_s1_mil1

    4 billion

    0.004435

    * | select avg(double_0), sum(double_0), max(double_0), min(double_0), count(double_0) from stress_s1_mil1

    4 billion

    0.006504

    * | select avg(double_0), sum(double_1), max(double_2), min(double_3), count(double_4) from stress_s1_mil1

    4 billion

    0.013600

    * | select key_0 , avg(double_0) as pv from stress_s1_mil1 group by key_0 order by pv desc limit 1000

    4 billion

    0.011826

    * | select long_0, avg(double_0) as pv from stress_s1_mil1 group by long_0 order by pv desc limit 1000

    4 billion

    0.011087

    * | select long_0, long_1, avg(double_0) as pv from stress_s1_mil1 group by long_0,long_1 order by pv desc limit 1000

    0.3 billion

    0.010791

    * | select avg(double_0) from stress_s1_mil1 where key_0='key_987'

    4 billion

    0.00007