:Downsampling

If metric data is collected at a high sampling rate, you can perform downsampling to reduce the sampling rate. You can perform downsampling to reduce the volume of data that you want to query and extend the data retention period. This way, the query speed can be increased and most trends and characteristics of raw data can be retained. Simple Log Service provides the downsampling feature to group and aggregate the data points in a time series based on a specific algorithm.

Limits

The number of active time series (time series exist within the maximum aggregation period) in a Metricstore must be less than the value calculated by using the following formula: Number of readwrite shards × 1 million. Otherwise, the data in the corresponding Metricstore for downsampling is incomplete.

Implementation

After you create a downsampling configuration, Simple Log Service creates a dedicated Metricstore based on the specified retention period and a dedicated Scheduled SQL job based on the specified aggregation period. You can create multiple downsampling configurations based on your business requirements. The downsampled data in the dedicated Metricstore of the first downsampling configuration is written from the original Metricstore. The downsampled data in the dedicated Metricstore of a subsequent downsampling configuration is written from the dedicated Metricstore of the previous downsampling configuration. If you modify a downsampling configuration, Simple Log Service automatically creates, modifies, or deletes the corresponding Metricstore or Scheduled SQL job based on the modified settings.

If the creation time and aggregation period of a downsampling configuration matches your query time range, the corresponding Metricstore is used as the data source of your query. This way, you can obtain data that is retained for a longer period of time and increase the query efficiency. In this example, the Metricstore of Downsampling Configuration 1 is used as the data source when you set the query step to 30 minutes, as shown in the following figure.

Assets

After you create a downsampling configuration for a Metricstore, Simple Log Service creates a dedicated Metricstore and a dedicated Scheduled SQL job in the project to which the original Metricstore belongs.

• Metricstore

  A dedicated Metricstore is used to store downsampled data. After you create a downsampling configuration, Simple Log Service automatically creates a dedicated Metricstore named {metricstore}--ds-{config-no}.

• Scheduled SQL job

  A dedicated Schedule SQL job is used to import downsampled data to the corresponding Metricstore at a specific interval. After you create a downsampling configuration, Simple Log Service automatically creates a dedicated Scheduled SQL job named {metricstore}_{config-no - 1}_to_{config-no}_downsampling.

Create a downsampling configuration

1. Log on to the Simple Log Service console.

2. In the Projects section, click the project that you want to manage.

   

3. On the Time Series Storage > Metricstore tab, find the Metricstore that you want to manage and click the icon to go to the Metricstore Attribute page.

4. In the upper-right corner of the page, click Modify. In the Downsampling Settings section, click Configure.

5. In the Downsampling Settings page, configure the parameters. The following table describes the parameters.

   Parameter	Description
   Metric Interval in Metricstore(Seconds)	The collection interval between two identical time series in the original Metricstore. You can configure this parameter based on your business scenario or click Automatic Calculation. If you select the automatic calculation method, the system randomly extracts a time series from the current Metricstore, calculates the interval of the metric, and then returns an interval rounded to a multiple of 5 seconds.
   Retention Period in Metricstore(Days)	The data retention time of the original Metricstore. If you change the data retention period of the original Metricstore on the Metricstore Attribute page, the value of the Retention Period in Metricstore(Days) parameter is automatically synchronized. You do not need to change the value. If the value of Retention Period in Metricstore(Days) is different from the value of Data Retention Period on the Metricstore Attribute page, click Synchronize.
   Downsampling Settings	You can add up to 5 downsampling configurations for a Metricstore. You can manually configure downsampling-related parameters or use the automatic generation method. Manual configuration Aggregation Period: the metric interval of the Metricstore for the current downsampling configuration. Important The value of the Aggregation Period parameter must be greater than or equal to 60 and cannot be empty. The aggregation period of the current configuration must be divided by the aggregation period of the previous configuration or the metric interval of the Metricstore. The aggregation period of the current configuration must be two or more times longer than the aggregation period of the previous configuration or the metric interval of the Metricstore. Retention Period: the data retention period of the Metricstore for the current downsampling configuration. Important The value of the Retention Period parameter must be greater than 0 and cannot be empty. The data retention period of the current configuration must be greater than or equal to the data retention period of the previous configuration. Auto generation: Click Recommended Downsampling Configuration to automatically generate downsampling configurations based on the current metric interval and data retention period. Important If you click Recommended Downsampling Configuration, the downsampling configurations that are not submitted are overwritten.

6. Click Submit to save the configurations.

Delete downsampling configurations

1. In the upper-right corner of the Metricstore Attribute page, click Modify and click Configure next to Downsampling Settings.

2. Click Delete Downsampling Configurations to disable the downsampling feature.

Handle exceptions related to downsampling tasks

If an exception occurs when a downsampling task runs, you can view the status and handle the exception.

1. On the Metricstore Attribute page, click View next to Downsampling Settings.

2. In the Downsampling Settings dialog box, click the icon to view the cause.

   Important

   If the status of the downsampling task is RUNNING or FAILED, you cannot submit another downsampling task.

3. Click Retry to run the previous task, or click Delete Downsampling Configurations and then recreate downsampling configurations.