Simple Log Service:Query acceleration

Global cache

By default, the Prometheus Query compute engine does not cache query results. Each time you query data, all data is computed. Your query is inefficient in scenarios in which the data volume is large and the query time range is long. The global cache feature supports reuse of specific results for the same query. If the PromQL statement and the step parameter that you specify for a query are the same as the settings that you specify for a previous query, the two queries are considered the same, and the results of the previous query can be reused. The feature reuses the query results within the same query time range and separately queries data that is beyond the time range.

Concurrent computing

By default, the data that is obtained by executing a standard Prometheus query statement is computed on a server in a single coroutine. In scenarios in which multiple time series, long query time ranges, and complex computational logic are involved, the query speed is slow. Simple Log Service incorporates adaptions to the Prometheus engine to support concurrent and distributed Prometheus queries. The concurrent computing feature splits a PromQL query by time interval or time series and then schedules the query to multiple servers for execution. The performance of a query on multiple servers is improved by 2 to 10 times compared with a query on a single server.

• Splitting by time interval

  In this example, the execution interval of a query is 12 hours. The query is split into 6 subqueries for execution at 2-hour intervals. The 6 subqueries are concurrently executed for computing. The results of the subqueries are merged.

  query:    sum(metric)
  interval: 12h
  step:     2m

• Splitting by time series

  In this example, a metric contains 500,000 time series, and the number of global concurrent tasks is set to 10. A total of 10 tasks are concurrently executed. Each task computes the data related to 50,000 time series. After the tasks are executed, the results of the tasks are merged.

Configure the query acceleration settings of a Metricstore in the Simple Log Service console

On the Metricstore Attribute page, perform the following operations. For more information about how to access the Metricstore Attribute page, see Modify the configurations of a Metricstore.

You can configure concurrent computing in the following modes:

• In auto mode, Simple Log Service estimates the degree of concurrency for the current concurrent computing task based on the amount of data pulled by the same query in the results of the most recent query tasks.

• In static mode, you can manually configure the degree of concurrency. To configure concurrent computing in static mode, we recommend that you consult technical support of Simple Log Service.

Configure the MetricsConfig parameter of a Metricstore by using SDKs

Simple Log Service SDK for Java provides an operation that you can use to modify the MetricsConfig parameter. You can configure the fields for the MetricsConfig parameter in the JSON format. Example:

{
  "parallel_config": {
    "enable": true,
    "mode": "static",
    "parallel_count_per_host": 2,
    "time_piece_count": 8,
    "time_piece_interval": 21600,
    "total_parallel_count": 8
  },
  "query_cache_config": {
    "enable": true
  }
}

The following table describes the mappings between the parameters used to configure query acceleration, the fields of the MetricsConfig parameter in the JSON format, and the key fields in the FormValue parameter.

Global cache

Add x-sls-global-cache-enable=true to enable global cache.

https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query_range?query=sum(up)&start=1690876800&end=1690877800&step=10&x-sls-global-cache-enable=true

Concurrent computing

Add x-sls-parallel-enable=true&x-sls-parallel-count=16 to enable concurrent computing and set the degree of concurrency to 16.

https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query_range?query=sum(up)&start=1690876800&end=1690877800&step=10&x-sls-parallel-enable=true&x-sls-parallel-count=16