All Products
Search
Document Center

Simple Log Service:Manage a Logstore

Last Updated:Sep 03, 2024

This topic describes how to select, create, upgrade, and delete a Logstore. This topic also describes how to delete logs. Before you can delete a Logstore, you must delete all related Logtail configurations. If you want to delete logs, you can shorten the retention period of the logs.

Introduction to Logstores

A Logstore in Simple Log Service is used to collect, store, and query logs. A Logstore can belong to only one project. You can create multiple Logstores in a project.

Simple Log Service provides two types of Logstores: Standard Logstores and Query Logstores.

Important

Standard Logstores support the following billing modes:

  • Pay-by-ingested-data

  • Pay-by-feature

Query Logstores support only the pay-by-feature billing mode.

  • Standard Logstore: This type of Logstore supports the log analysis feature and is suitable for scenarios such as real-time monitoring and interactive analysis. You can use this type of Logstore to build a comprehensive observability system.

  • Query Logstore: The index traffic fee of a Query Logstore is lower than that of a Standard Logstore. Query Logstores do not support SQL analysis. Query Logstores are suitable for scenarios in which the volume of data is large, the log retention period is long, or log analysis is not required. Log retention periods of weeks or months are considered long.

For more information, see Logstore.

Limits

Logstores that are automatically created for features such as the service log and Object Storage Service (OSS) log collection features are dedicatedly used to store feature-specific logs. The Logstores include system Logstores and dedicated Logstores. You cannot write other logs to the Logstores. For more information, see Use the service log feature and Usage notes.

  • For example, internal-operation_log is a system Logstore that can store only the detailed logs of Simple Log Service.

  • For example, oss-log-store in a specified project is an OSS-dedicated Logstore that can store only the access logs of OSS.

Create a Logstore

Note

The number of Logstores that you can create in a project is limited. For more information, see Maximum Logstores.

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

    image

  3. On the Log Storage > Logstores tab, click the + icon.

    image

  4. In the Create Logstore panel, configure the following parameters and click OK.

    Parameter

    Description

    Logstore Type

    You can upgrade a Query Logstore to a Standard Logstore. You cannot downgrade a Standard Logstore to a Query Logstore.

    Important

    After you upgrade a Query Logstore to a Standard Logstore, Enable Analytics is automatically turned on for the existing fields. You can analyze only new logs that are written to Simple Log Service after the upgrade. Fees are not generated on the index traffic of historical data. For the index traffic of the new logs, fees are generated, and the involved billable item is the index traffic of log data. For more information, see Billable items of pay-by-feature.

    Billing Mode

    The billing mode of the Logstore. Valid values: Pay-by-ingested-data and Pay-by-feature. For more information, see Billable items.

    Logstore Name

    The name of the Logstore. The name must be unique in the project to which the Logstore belongs. After the Logstore is created, you cannot change the name of the Logstore.

    WebTracking

    If you turn on WebTracking, you can collect data from HTML, HTML5, iOS, and Android platforms to Simple Log Service by using the web tracking feature. For more information, see Use the web tracking feature to collect logs.

    Data Retention Period

    The retention period of logs in the Logstore.

    • If you set the Data Retention Period parameter to Specified Days, you can specify the number of days for which you want to store logs based on your business requirements. Valid values: 1 to 3650. Unit: days. After the retention period ends, logs are automatically deleted. If you set this parameter to 3650, data is permanently stored. If you change the retention period, Simple Log Service deletes all expired data within 1 hour. The data volume that is displayed for Usage Details on the homepage of the Simple Log Service console is updated the next day. For example, if you change the retention period from 5 days to 1 day, Simple Log Service deletes the logs of the previous four days within 1 hour.

    • If you set the Data Retention Period parameter to Permanent Storage, Simple Log Service permanently stores the collected logs in the Logstore.

    Note

    If you query the retention period by calling an API operation and the returned result is 3650, logs are permanently stored.

    Intelligent Tiered Storage

    For more information about how to enable intelligent tiered storage, see Configure intelligent tiered storage. For more information about the fees generated for data storage in the hot storage tier, Infrequent Access (IA) storage tier, and Archive storage tier, see Billable items.

    Automatic Sharding

    If you turn on Automatic Sharding, Simple Log Service increases the number of shards when the existing shards cannot accommodate the data that is written. For more information, see Manage shards.

    Maximum Shards

    If you turn on Automatic Sharding, you must configure this parameter to specify the maximum number of readwrite shards that can be created. Maximum value: 256.

    Log Public IP

    If you turn on Log Public IP, Simple Log Service adds the following information to the Tag field of the collected logs:

    • __client_ip__: the public IP address of the log source.

    • __receive_time__: the time at which Simple Log Service receives the log. The value is a UNIX timestamp representing the number of seconds that have elapsed since January 1, 1970, 00:00:00 UTC.

Modify the configurations of a Logstore

  1. On the Log Storage > Logstores tab, move the pointer over the Logstore whose configurations you want to modify and choose 修改日志库 > Modify.

    image

  2. On the Logstore Attributes page, click Modify. After you modify the parameters, click Save. For more information about the parameters, see Create a Logstore.

    image

Delete a Logstore

  1. Before deletion

    1. Before you can delete a Logstore, you must delete all Logtail configurations that are associated with the Logstore. For more information, see Delete Logtail configurations.

    2. If the log shipping feature is enabled for the Logstore, we recommend that you stop writing data to the Logstore and make sure that all data in the Logstore is shipped before you delete the Logstore.

  2. Deletion

    1. On the Log Storage > Logstores tab, move the pointer over the Logstore that you want to delete and choose 修改日志库 > Delete.

      Warning

      After you delete a Logstore, all logs in the Logstore are deleted and cannot be restored. Proceed with caution.

    2. In the Warning dialog box, enter the Logstore name and click Confirm Deletion.

  3. After deletion

    1. After you delete all Logstores, you are still charged for data storage on the day that you delete the Logstores. You are no longer charged for data storage starting the following day. Starting the third day after you delete all Logstores, you no longer receive a Simple Log Service bill.

API operations

Action

Operation

Create a Logstore

CreateLogStore

Delete a Logstore

DeleteLogStore

Query a Logstore

Modify a Logstore

UpdateLogStore

Appendix: Logstore comparison by type

If you set the Billing Mode parameter to Pay-by-feature, you can set the Logstore Type parameter to Standard or Query. The following table describes the differences between the two types of Logstores.

Item

Standard Logstore

Query Logstore

Fee

Index traffic

USD 0.0875 per GB

USD 0.0146 per GB

Feature

Collection of business system logs

Supported

Supported

Logs of cloud services cannot be collected.

Intelligent tiered storage

Supported

Supported

Log search

Supported

Supported

Log analysis based on SQL statements

Supported

Not supported

Contextual query

Supported

Supported

LiveTail

Supported

Supported

LogReduce

Supported

Not supported

Reindexing

Supported

Supported

Dashboard

Supported

Not supported

Alerting

Supported

Supported only based on search statements

Scheduled SQL

Supported

Not supported

Data transformation

Supported

Supported

Data shipping

Supported

Supported

Data consumption

Supported

Supported