Data in Log Service is stored in shards. This topic describes how to split, merge, and delete shards in the Log Service console.

Background information

Log Service uses shards to manage the read and write capabilities of Logstores or Metricstores. When you create a Logstore or a Metricstore, you must specify the number of shards for the Logstore or Metricstore. After the Logstore or Metricstore is created, you can split or merge shards to increase or decrease the number of shards in the Logstore or Metricstore. For more information, see Shard.
  • Each shard supports a write speed of up to 5 MB/s and a read speed of up to 10 MB/s. If the read speed or write speed of a shard cannot meet your business requirements, we recommend that you split the shard.

    You can split a shard of a Logstore on the Logstore Attributes page of the Logstore.

  • If the data traffic is much smaller than the maximum read speed or write speed of a shard, we recommend that you merge the shard.

Split a shard

  1. Log on to the Log Service console.
  2. In the Projects section, click the project that you want to view.
  3. On the Log Storage > Logstores tab, find the Logstore whose configurations you want to modify and choose Modify a Logstore > Modify.
  4. On the Logstore Attributes page, click Modify.
  5. Find the shard that you want to split, and click Split in the Actions column.
    Important You can split only a shard that is in the readwrite state.
    Split a shard
  6. Select the number of new shards into which you want to split the original shard, and click OK.
  7. Click Save.

    For example, if you set the number of new shards to 2, two shards whose status is readwrite are added. The status of the original shard changes from readwrite to readonly. New shards are displayed below the original shard. The MD5 hash ranges of the new shards cover the MD5 hash range of the original shard. You can consume data from the original shard. However, you cannot write data to the original shard. For more information, see Shard.

Automatic sharding

Log Service provides the automatic sharding feature. After you enable the automatic sharding feature, shards are automatically split if the following conditions are met:
  • The data write speed exceeds the maximum write speed of the current shard for more than 5 minutes.
  • The number of shards that are in the readwrite state does not exceed the specified maximum number of shards in the Logstore.
Note Automatic sharding does not apply to the shards that are split from a shard in the last 15 minutes.
You can enable the automatic sharding feature when you create or modify a Logstore. If you turn on Automatic Sharding, you must configure the Maximum Shards parameter.
  • Automatic Sharding

    For example, Automatic Sharding is turned on, and the data write speed exceeds the maximum write speed of the current shard for more than 5 minutes. In this case, the shard is automatically split.

    For example, a Logstore contains four shards. Log Service checks whether each shard meets the splitting conditions. The shards that meet the splitting conditions are separately split. The total number of readwrite shards that are obtained after shards are automatically split does not exceed the specified maximum number of shards.

  • Maximum Shards

    The maximum number of readwrite shards that can be obtained after shards are automatically split. If you enable the automatic sharding feature, a maximum of 64 readwrite shards can be obtained after shards are automatically split.

Merge shards

You can merge the shards of a Logstore. If you click Merge in the Actions column of a shard, Log Service locates the shard next to the specified shard, and merges the two shards.

Note When you merge shards, you must specify a shard that is in the readwrite state. However, you cannot specify the last shard that is in the readwrite state.
  1. On the Log Storage > Logstores tab, find the Logstore whose configurations you want to modify and choose Modify a Logstore > Modify.
  2. On the Logstore Attributes page, click Modify.
  3. Find the shard that you want to merge and click Merge in the Actions column.
    Merge shards
  4. Click Save.
    After you merge shards, the specified shard and the shard next to the specified shard are in the readonly state. A new shard is generated and is in the readwrite state. The MD5 hash range of the new shard covers the MD5 hash ranges of the original shards.

Delete a shard

  • Automatic deletion

    You can specify a data retention period when you create a Logstore. If you specify the retention period, the shards in the Logstore and the data stored in the shards are automatically deleted when the retention period elapses.

  • Manual deletion

    If you turn on Permanent Storage when you create a Logstore, we recommend that you delete the shards and data of the Logstore by deleting the Logstore. For more information, see Delete a Logstore.

Shard-related API operations

ActionOperation
Split a shardSplitShard
Merge shardsMergeShards
Query shardsListShards