All Products
Search
Document Center

Simple Log Service:Manage a Logstore

Last Updated:Jan 16, 2024

This topic describes how to select, create, upgrade, and delete a Logstore. This topic also describes how to delete logs. Before you can delete a Logstore, you must delete all related Logtail configurations. If you want to delete logs, you can reduce the retention period for the logs.

Terms

A Logstore in Simple Log Service is used to collect, store, and query logs. Each Logstore belongs to a project. You can create multiple Logstores in a project.

Simple Log Service provides two types of Logstores: Standard Logstores and Query Logstores.

Important

If you set Billing Mode to Pay-by-ingested-data, you can set Logstore Type only to Standard.

  • Standard Logstore: This type of Logstore supports the log analysis feature and is suitable for scenarios such as real-time monitoring and interactive analysis. You can use this type of Logstore to build a comprehensive observability system.

  • Query Logstore: This type of Logstore supports high-performance queries. The index traffic fee of a Query Logstore is lower than that of a Standard Logstore. Query Logstores do not support SQL analysis. Query Logstores are suitable for scenarios in which the volume of data is large, the log retention period is long, or log analysis is not required. Log retention periods of weeks or months are considered long.

For more information, see Logstore.

Logstore comparison by type

If you set Billing Mode to Pay-by-feature, you can set Logstore Type to Standard or Query. The following table describes the differences between the two types of Logstores.

Item

Standard Logstore

Query Logstore

Fee

Index traffic

USD 0.0875 per GB

USD 0.0146 per GB

Feature

Collection of business system logs

Supported

Supported.

Logs of cloud services cannot be collected.

Intelligent tiered storage

Supported

Supported

Log search

Supported

Supported

Log analysis based on SQL statements

Supported

Not supported

Context query

Supported

Supported

LiveTail

Supported

Supported

LogReduce

Supported

Not supported

Reindexing

Supported

Supported

Dashboard

Supported

Not supported

Alerting

Supported

Alerts can be configured based only on search statements.

Scheduled SQL

Supported

Not supported

Data transformation

Supported

Supported

Data shipping

Supported

Supported

Data consumption

Supported

Supported

Limits

System Logstores and dedicated Logstores for cloud services cannot store irrelevant logs.

  • For example, internal-operation_log is a system Logstore that can store only the detailed logs of Simple Log Service.

  • For example, oss-log-store in a specified project is a dedicated Logstore for Object Storage Service (OSS) that can store only the access logs of OSS.

Create a Logstore

Note

You can create up to 200 Logstores in a project.

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the + icon.

  4. In the Create Logstore panel, configure the following parameters.

  5. Parameter

    Description

    Logstore Type

    The type of the Logstore. Simple Log Service provides two types of Logstores: Standard Logstores and Query Logstores. For more information about the differences between the two types of Logstores, see Logstore comparison by type.

    You can upgrade a Query Logstore to a Standard Logstore. For more information, see Method 1.

    Important

    You cannot downgrade a Standard Logstore to a Query Logstore.

    Billing Mode

    The billing mode of the Logstore. Valid values: Pay-by-ingested-data and Pay-by-feature. For more information, see Billable items.

    Note

    The first time you use the pay-by-ingested-data billing mode, you must enable the billing mode as prompted.

    Logstore Name

    The name of the Logstore. The name must be unique in the project to which the Logstore belongs. After the Logstore is created, you cannot change the name of the Logstore.

    WebTracking

    If you turn on WebTracking, you can collect data from HTML, HTML5, iOS, and Android platforms to Simple Log Service by using the web tracking feature. For more information, see Use the web tracking feature to collect logs.

    Data Retention Period

    The retention period of logs in the Logstore.

    • If you select Specified Days for Data Retention Period, you can specify how many days you want to store logs based on your business requirements. Valid values: 1 to 3000. Unit: days.

      Warning
      • After the retention period ends, logs are automatically deleted.

      • If you shorten the data retention period, Simple Log Service deletes all expired data within 1 hour. The data volume that is displayed for Usage Details on the homepage of the Simple Log Service console is updated the next day. For example, if you change the data retention period from 5 days to 1 day, Simple Log Service deletes the logs of the previous four days within 1 hour.

    • If you select Permanent Storage for Data Retention Period, Simple Log Service permanently stores the collected logs in the Logstore.

      Note

      If you query the data retention period by calling an API operation and the returned result is 3650, logs are permanently stored.

    Intelligent Tiered Storage

    If you turn on Intelligent Tiered Storage, you can store data in the hot storage tier, Infrequent Access (IA) storage tier, or Archive storage tier based on your business requirements.

    • You can specify a retention period for data that is stored in the hot storage tier and select Automatically Converted to IA. In this case, data is first stored in the hot storage tier and then automatically moved to the IA storage tier after the specified retention period. If you specify a retention period for data that is stored in the IA storage tier and select Automatically Deleted, data that is stored in the IA storage tier is automatically deleted after the specified retention period.

    • You can specify a retention period for data that is stored in the hot storage tier and select Automatically Converted to IA. In this case, data is first stored in the hot storage tier and then automatically moved to the IA storage tier after the specified retention period. If you specify a retention period for data that is stored in the IA storage tier and select Automatically Converted to Archive, data is first stored in the IA storage tier and then automatically moved to the Archive storage tier after the specified retention period.

    • You can specify a retention period for data that is stored in the hot storage tier and select Automatically Converted to Archive. In this case, data is first stored in the hot storage tier and then automatically moved to the Archive storage tier after the specified retention period. If you specify a retention period for data that is stored in the Archive storage tier and select Automatically Deleted, data that is stored in the Archive storage tier is automatically deleted after the specified retention period.

    Important
    • For more information about the hot storage tier, IA storage tier, and Archive storage tier, see Overview of tiered storage.

    • Data in the hot storage tier must be stored for at least 7 days before it can be moved to the IA storage tier and at least 30 days before it can be moved to the Archive storage tier. Data in the IA storage tier must be stored for at least 30 days before it can be moved to the Archive storage tier. For more information, see Lifecycle management for data storage.

    • For more information about the billing of data in the hot storage tier, IA storage tier, and Archive storage tier, see Billable items.

    Shards

    The number of shards. Simple Log Service provides shards that allow you to read and write data. Each shard supports a write capacity of 5 MB/s and 500 writes/s and a read capacity of 10 MB/s and 100 reads/s. You can create up to 10 shards in each Logstore. You can create up to 200 shards in each project. For more information, see Shard.

    By default, two shards are created when you create a Logstore. For more information about the billing of shards, see Why am I charged for active shards?

    Automatic Sharding

    If you turn on Automatic Sharding, Simple Log Service increases the number of shards when the existing shards cannot accommodate the data that is written. For more information, see Manage shards.

    Maximum Shards

    If you turn on Automatic Sharding, you must configure this parameter to specify the maximum number of readwrite shards that can be created. Maximum value: 256.

    Log Public IP

    If you turn on Log Public IP, Simple Log Service adds the following information to the Tag field of the collected logs:

    • __client_ip__: the public IP address of the log source.

    • __receive_time__: the time at which Simple Log Service receives the log. The value is a UNIX timestamp representing the number of seconds that have elapsed since the epoch time January 1, 1970, 00:00:00 UTC.

  6. Click OK.

Modify the configurations of a Logstore

  1. On the Log Storage > Logstores tab, move the pointer over the Logstore whose configurations you want to modify and choose 修改日志库 > Modify.

  2. On the Logstore Attributes page, click Modify.

    For more information about the parameters, see Create a Logstore.

    Important

    You can switch the billing mode of a Logstore from pay-by-ingested-data to pay-by-feature. You cannot switch the billing mode of a Logstore from pay-by-feature to pay-by-ingested-data.

  3. Click Save.

Upgrade the specifications of a Logstore

You can upgrade a Query Logstore to a Standard Logstore. You can upgrade the specifications of a Logstore on the Logstore Attributes page. You can also follow the on-screen instructions to upgrade the specifications of a Logstore when you perform analysis operations.

Important
  • After the upgrade, Enable Analytics is automatically turned on for the current fields. You can analyze only the logs that are written to Simple Log Service after the log analysis feature is enabled. You can turn on Enable Analytics to enable the feature.

    If you enable the log analysis feature, fees are not generated for the index traffic of historical data.

  • After the upgrade, you are charged for the index traffic of new logs that are written to Simple Log Service. The index traffic fee is calculated based on the index traffic of log data billable item. For more information, see Billable items of pay-by-feature.

Method 1

  1. On the Log Storage > Logstores tab, move the pointer over the Logstore whose specifications you want to upgrade and choose 修改日志库 > Modify.

  2. On the Logstore Attributes page, click Modify.

  3. Find Logstore Type and click Upgrade.

  4. In the message that appears, click Confirm.

  5. Click Save.

Method 2

When you configure an index for a Logstore, click the switch in the Enable Analytics column. Then, you are prompted to upgrade the specifications of the Logstore. You can click Upgrade Now to complete the upgrade. For more information about index configurations, see Create indexes. 升级Logstore规格

Method 3

On the Quick Analysis tab of a Logstore, click the field that you want to manage. Then, you are prompted to upgrade the specifications of the Logstore. You can click Upgrade Now to complete the upgrade. For more information about the quick analysis feature, see Quick analysis.升级Logstore规格

Delete a Logstore

Important
  • Before you can delete a Logstore, you must delete all Logtail configurations that are associated with the Logstore. For more information, see Delete Logtail configurations.

  • If the log shipping feature is enabled for the Logstore, we recommend that you stop writing data to the Logstore and make sure that all data in the Logstore is shipped before you delete the Logstore.

  • If you delete all Logstores, you are still charged for data storage on the day that you delete the Logstores, but no charges are incurred from the following day. As a result, from the third day after you delete all Logstores, you no longer receive a Simple Log Service bill.

  1. On the Log Storage > Logstores tab, move the pointer over the Logstore that you want to delete and choose 修改日志库 > Delete.

    Warning

    After you delete a Logstore, all logs in the Logstore are deleted and cannot be restored. Proceed with caution.

  2. In the Delete message, click OK.

Delete logs

After the retention period ends, logs are automatically deleted. You can also modify Data Retention Period to delete logs.

Important

If you shorten the data retention period, Simple Log Service deletes all expired data within 1 hour. The data volume that is displayed for Usage Details on the homepage of the Simple Log Service console is updated the next day. For example, if you change the data retention period from 5 days to 1 day, Simple Log Service deletes the logs of the previous four days within 1 hour.

Enable the intelligent tiered storage feature

You can enable the intelligent tiered storage feature for a Logstore on the Logstore Attributes page. For more information, see Enable intelligent tiered storage.

Logstore-related API operations

Action

API operation

Create a Logstore

CreateLogStore

Delete a Logstore

DeleteLogStore

Query a Logstore

Modify a Logstore

UpdateLogStore

References

To aggregate data in multiple Logstores, you can use the data transformation feature. For more information, see Aggregate data from multiple source Logstores.