All Products
Search
Document Center

Simple Log Service:Manage shards

Last Updated:Jan 25, 2024

You can split or merge shards to adjust the number of active shards, which determine the maximum read and write capacities of Simple Log Service Logstores. A shard can be automatically split. However, you must manually merge shards. This topic describes how to split, merge, and delete shards. This topic also describes how to configure automatic sharding.

Term

Data in Simple Log Service is stored in shards. Simple Log Service uses shards to manage the read and write capacities of Logstores, Eventstores, and Metricstores. When you create a Logstore, a Metricstore, or an Eventstore, you must specify the number of shards. After the Logstore, Metricstore, or Eventstore is created, you can split or merge shards to increase or decrease the number of shards. For more information, see Shard.

Split a shard

We recommend that you plan the number of shards based on your actual data traffic. Each shard supports a write speed of up to 5 MB/s or 500 write operations per second, and a read speed of up to 10 MB/s or 100 read operations per second. The limits are not hard limits. If a limit is exceeded, Simple Log Service continues to read or write data. However, performance may be degraded. If the read and write traffic of data exceeds the read or write capacity of a shard, you can split the shard into multiple shards to increase the capacity.

Note

If the error code 403 or 500 is frequently returned when you write data by calling Simple Log Service API, you can go to the CloudMonitor console to check the traffic and status codes. Then, you can determine whether to increase the number of shards.

Console operations

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, move the pointer over the Logstore whose configurations you want to modify and choose 修改日志库 > Modify.

  4. On the Logstore Attributes page, click Modify.

  5. Find the shard that you want to split and click Split in the Actions column.

    Important

    You can split only a shard that is in the readwrite state.

    分裂分区

  6. Select the number of new shards into which you want to split the original shard, and click OK.

  7. Click Save.

Automatic sharding

Simple Log Service provides the automatic sharding feature to automatically handle business traffic that exceeds the estimated value. Shards are automatically split only if the following conditions are met:

  • Automatic Sharding is turned on.

  • The data write speed exceeds the maximum write speed of the current shard for more than 5 minutes.

  • The number of shards that are in the readwrite state does not exceed the specified maximum number of shards in your Logstore.

Note

Automatic sharding is not performed on the shards that are split from a shard within the previous 15 minutes.

You can enable the automatic sharding feature when you create or modify a Logstore. If you turn on Automatic Sharding, you must configure the Maximum Shards parameter.

  • Automatic Sharding

    For example, a Logstore contains four shards. Simple Log Service checks whether each shard meets the splitting conditions. The shards that meet the splitting conditions are split. The total number of new shards does not exceed the specified maximum number of shards.

  • Maximum Shards

    This parameter specifies the maximum number of new shards that can be obtained after shards are automatically split. After you enable the automatic sharding feature, up to 256 shards that are in the readwrite state can be obtained after splitting.

Merge shards

If the read and write traffic of data is very small compared with the maximum read and write capacities of your shard, we recommend that you merge shards to reduce the fees of active shards. You can find your shard and click Merge in the Actions column to merge shards. Simple Log Service automatically finds a shard that is adjacent to your shard and merges the two shards. Two shards are considered adjacent if the MD5 hash ranges for the shards form a contiguous range with no gaps. Shards can only be manually merged.

Important

When you merge shards, you must specify a shard that is in the readwrite state. You cannot specify the last shard that is in the readwrite state.

  1. On the Log Storage > Logstores tab, move the pointer over the Logstore whose configurations you want to modify and choose 修改日志库 > Modify.

  2. On the Logstore Attributes page, click Modify.

  3. Find the shard that you want to merge and click Merge in the Actions column.

    合并分区

  4. Click Save.

    After you merge shards, the specified shard and its adjacent shard are in the readonly state. A new shard that is in the readwrite state is generated. The MD5 hash range of the new shard covers the MD5 hash ranges of the original shards.

Delete a shard

Warning

After a shard is deleted, the shard cannot be restored. Proceed with caution.

  • Automatic deletion

    If you specify a data retention period when you create a Logstore, the shards in the Logstore and the data that is stored in the shards are automatically deleted when the retention period ends.

  • Manual deletion

    If you select Permanent Storage for Data Retention Period when you create a Logstore, we recommend that you delete the shards and data of the Logstore by deleting the Logstore. For more information, see Delete a Logstore.

Shard-related API operations

Action

API operation

Split a shard

SplitShard

Merge shards

MergeShard

Query shards

ListShards