Overview

Last Updated: Jul 25, 2017

Logstore read/write logs must be saved in a certain shard. Each Logstore is divided into several shards and each shard is composed of MD5 left-closed, right-open intervals. These intervals do not overlap and the ranges of all intervals equal the entire MD5 value range.

Range

When creating a Logstore, the entire MD5 range will be automatically divided into even sets, based on the specified number of shards. Each shard has a certain range within the range of [00000000000000000000000000000000,ffffffffffffffffffffffffffffffff).

A shard is an MD5 left-closed, right-open interval, and is composed of the following keys.

  • BeginKey: The range key is inclusive of the shard range, indicating the start of the shard.
  • EndKey: The range key is exclusive of the shard range, indicating the end of the shard.

Shard range is used to support writing logs by specifying hash key, as well as for shard split and merge operations. When reading data from a shard, the corresponding shard must be specified. When writing data, you can use load balancing mode or specify hash key. In load balancing mode, each packet is written to an available shard at random. By specifying hash key, data is written to the shard whose range includes the specified key.

Assume that the MD5 value range of a Logstore is from 00 to ff, and it has a total of 4 shards with the following ranges.

Shard No. Range
Shard0 [00,40)
Shard1 [40,80)
Shard2 [80,C0)
Shard3 [C0,FF)

If you write a log and specify the MD5 key 5F, this request will go to Shard1. If you specify the MD5 key 8C, this request will go to Shard2.

Shard read/write capacities

Each shard has certain service capacities:

  • Write: 5 MB/s, 2000 times/s
  • Read: 10 MB/s, 100 times/s

You can calculate the number of shards needed based on the traffic. If you have already set up several shards, you can split or merge these shards to increase or decrease the number.

  • When writing logs, if the API consistently reports error code 403 or 500, use the Logstore CloudMonitor to view the traffic and status code and determine whether you need to increase the number of shards.
  • For read/write operations that exceed a shard’s service capacities, the system runs according to your required services, however, the service quality cannot be guaranteed as being optimal.

Shard status

  • readwrite: Capable of reading and writing
  • readonly: Read-only data

Shard operations

On the Log Service console, you can perform the following shard operations:

Thank you! We've received your feedback.