A Logstore in Log Service is used to collect, store, and query logs. This topic describes the types of Logstores and the operations that you can perform on a Logstore.
- Standard Logstore: This type of Logstore supports the log analysis feature and is suitable for scenarios such as real-time monitoring and interactive analysis. You can use this type of Logstore to build a comprehensive observability system.
- Query Logstore: This type of Logstore supports high-performance queries. The index traffic fee of a Query Logstore is lower than that of a Standard Logstore. Query Logstores do not support SQL analysis. Query Logstores are suitable for scenarios in which the volume of data is large, the log retention period is long, or log analysis is not required. Log retention periods of weeks or months are considered long.
Logstore comparison by type
|Item||Standard Logstore||Query Logstore|
|Fee||Index traffic||USD 0.0875 per GB||USD 0.0146 per GB|
|Feature||Collection of business system logs||Supported||Supported|
Logs of cloud services cannot be collected.
|Enable hot and cold-tiered storage for a Logstore||Supported||Supported|
|Log analysis based on SQL statements||Supported||Not supported|
|Reindex logs for a Logstore||Supported||Supported|
|Alerting||Supported||Alerts can be configured based only on search statements.|
|Scheduled SQL||Supported||Not supported|
- For example, internal-operation_log is a system Logstore that can store only the detailed logs of Log Service.
- For example, oss-log-store in a specified project is a dedicated Logstore for Object Storage Service (OSS) that can store only the access logs of OSS.
Create a Logstore
- Log on to the Log Service console.
- In the Projects section, click the project that you want to manage.
- On the + icon. tab, click the
- In the Create Logstore panel, configure the following parameters.
Parameter Description Logstore Type The type of the Logstore. Log Service provides two types of Logstores: Standard Logstores and Query Logstores. For more information about the differences between the two types of Logstores, see Logstore comparison by type.
You can upgrade a Query Logstore to a Standard Logstore. For more information, see Method 1.
Logstore Name The name of the Logstore. The name must be unique in the project to which the Logstore belongs. After the Logstore is created, you cannot change its name. WebTracking If you turn on WebTracking, you can collect data from HTML, HTML5, iOS, and Android platforms to Log Service by using the web tracking feature. For more information, see Use the web tracking feature to collect logs. Permanent Storage If you turn on Permanent Storage, Log Service permanently stores the collected logs in the Logstore.Note If you query the data retention period by calling an API operation and the returned result is 3650, logs are permanently stored. Data Retention Period The retention period of logs in the Logstore. Valid values: 1 to 3000. Unit: days.Warning Logs are automatically deleted after the specified retention period ends.
You can configure the Data Retention Period parameter only if you do not turn on Permanent Storage.Important If you shorten the data retention period, Log Service deletes all expired data within 1 hour. The data volume that is displayed for Storage Size(Log) on the homepage of the Log Service console is updated the next day. For example, if you change the value of the Data Retention Period parameter from 5 to 1, Log Service deletes the logs of the previous four days within 1 hour.
Shards The number of shards. Log Service provides shards that allow you to read and write data. Each shard supports a write capacity of 5 MB/s and 500 writes/s and a read capacity of 10 MB/s and 100 reads/s. You can create up to 10 shards in each Logstore. You can create up to 200 shards in each project. For more information, see Shard:
By default, two shards are created when you create a Logstore. For more information about the billing of shards, see Why am I charged for active shards? .
Automatic Sharding If you turn on Automatic Sharding, Log Service increases the number of shards when the existing shards cannot accommodate the data that is written. For more information, see Manage shards. Maximum Shards If you turn on Automatic Sharding, you must configure this parameter to specify the maximum number of readwrite shards that can be created. Maximum value: 256. Log Public IP If you turn on Log Public IP, Log Service adds the following information to the Tag field of the collected logs:
- __client_ip__: the public IP address of the log source.
- __receive_time__: the time at which Log Service receives the log. The value is a UNIX timestamp representing the number of seconds that have elapsed since the epoch time January 1, 1970, 00:00:00 UTC.
- Click OK.
Modify the configurations of a Logstore
- On the tab, move the pointer over the Logstore whose configurations you want to modify and choose .
- On the Logstore Attributes page, click Modify. For more information about the parameters, see Create a Logstore.
- Click Save.
Upgrade the specifications of a Logstore
You can upgrade a Query Logstore to a Standard Logstore. You can upgrade the specifications of a Logstore on the Logstore Attributes page. You can also follow the on-screen instructions to upgrade the specifications of a Logstore when you perform analysis operations.
- After the upgrade, Enable Analytics is automatically turned on for the current fields. You can analyze only the logs that are written to Log Service after the log analysis feature is enabled. You can turn on Enable Analytics to enable the feature.
If you enable the log analysis feature, fees are not generated for the index traffic of historical data.
- After the upgrade, you are charged for the index traffic of new logs that are written to Log Service. The index traffic fee is calculated based on the index traffic of log data billable item. For more information, see Billable items.
- On the tab, move the pointer over the Logstore whose specifications you want to upgrade and choose .
- On the Logstore Attributes page, click Modify.
- Find Logstore Type and click Upgrade.
- In the message that appears, click Confirm.
- Click Save.
Delete a Logstore
- Before you can delete a Logstore, you must delete all Logtail configurations that are associated with the Logstore. For more information, see Delete Logtail configurations.
- If the log shipping feature is enabled for the Logstore, we recommend that you stop writing data to the Logstore and make sure that all data in the Logstore is shipped before you delete the Logstore.
- If you delete all Logstores, you are still charged for data storage on the day that you delete the Logstores, but no charges are incurred from the following day. As a result, from the third day after you delete all Logstores, you no longer receive a Log Service bill.
- On the tab, move the pointer over the Logstore that you want to delete and choose . Warning After you delete a Logstore, all logs in the Logstore are deleted and cannot be restored. Proceed with caution.
- In the Delete message, click OK.
Logs are automatically deleted after the specified retention period ends. To delete logs, you can modify Data Retention Period.
Enable hot and cold-tiered storage for a Logstore
You can enable the hot and cold-tiered storage feature for a Logstore on the Logstore Attributes page in the Log Service console. For more information, see Enable hot and cold-tiered storage for a Logstore.
Logstore-related API operations
|Create a Logstore||CreateLogStore|
|Delete a Logstore||DeleteLogStore|
|Query a Logstore|
|Modify a Logstore||UpdateLogStore|