Use Simple Log Service SDK for Java to manage shards - Simple Log Service

Shards are used to manage the read and write capacity of Logstores or Metricstores. In Simple Log Service, data is stored in a shard. This topic describes how to query, split, and merge shards by using Simple Log Service SDK for Java and provides sample code.

Prerequisites

A Resource Access Management (RAM) user is created, and the required permissions are granted to the RAM user. For more information, see Create a RAM user and grant permissions to the RAM user.
The ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables are configured. For more information, see Configure environment variables in Linux, macOS, and Windows.
Important
- The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. We recommend that you use the AccessKey pair of a RAM user to call API operations or perform routine O&M.
- We recommend that you do not save the AccessKey ID or AccessKey secret in your project code. Otherwise, the AccessKey pair may be leaked, and the security of all resources within your account may be compromised.
Simple Log Service SDK for Java is installed. For more information, see Install Simple Log Service SDK for Java.
A Logstore is created. For more information, see Sample code that is used to create a Logstore.
For the split and merge operations, the shard status must be readwrite.

Usage notes

In this example, the public Simple Log Service endpoint for the China (Hangzhou) region is used, which is https://cn-hangzhou.log.aliyuncs.com. If you want to access Simple Log Service by using other Alibaba Cloud services that reside in the same region as your project, you can use the internal Simple Log Service endpoint, which is https://cn-hangzhou-intranet.log.aliyuncs.com. For more information about the supported regions and endpoints of Simple Log Service, see Endpoints.

Sample code that is used to query shards

The following sample code provides an example on how to query the shards of a Logstore:

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Shard;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.ListShardResponse;

public class ListShards {
    public static void main(String[] args) throws LogException {
        // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // The name of the project. 
        String projectName = "ali-test-project";
        // The name of the Logstore. 
        String logstoreName = "ali-test-logstore";
        // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint. 
        String host = "https://cn-hangzhou.log.aliyuncs.com";

        // Create a Simple Log Service client. 
        Client client = new Client(host, accessId, accessKey);

        try {
            System.out.println("ready to list shards");

            // Query all shards of the Logstore. 
            ListShardResponse response = client.ListShard(projectName, logstoreName);

            for(Shard shard : response.GetShards()){
                System.out.println("Shard ID is :" + shard.getShardId());
                System.out.println("Shard status is :" + shard.getStatus());
                System.out.println("Shard begin key is :" + shard.getInclusiveBeginKey());
                System.out.println("Shard end key is :" + shard.getExclusiveEndKey());
                System.out.println("Shard create time is :" + shard.getCreateTime());
            }

            System.out.println(String.format("list shards of %s success", logstoreName));

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Expected result:

ready to list shards
Shard ID is :0
Shard status is :readwrite
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard create time is :1667810747
Shard ID is :1
Shard status is :readwrite
Shard begin key is :7f000000000000000000000000000000
Shard end key is :ffffffffffffffffffffffffffffffff
Shard create time is :1667810747
list shards of ali-test-logstore success

Sample code that is used to split a shard

The following sample code provides an example on how to split a shard of a Logstore:

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Shard;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.ListShardResponse;

import java.util.concurrent.TimeUnit;

public class SplitShard {
    public static void main(String[] args) throws LogException {
        // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // The name of the project. 
        String projectName = "ali-test-project";
        // The name of the Logstore. 
        String logstoreName = "ali-test-logstore";
        // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint. 
        String host = "https://cn-hangzhou.log.aliyuncs.com";

        // Create a Simple Log Service client. 
        Client client = new Client(host, accessId, accessKey);

        try {
            System.out.println("ready to list shard");

            // Query all shards before splitting. 
            ListShardResponse response = client.ListShard(projectName, logstoreName);
            for(Shard shard : response.GetShards()){
                System.out.println("Shard ID is :" + shard.getShardId());
                System.out.println("Shard status is :" + shard.getStatus());
                System.out.println("Shard begin key is :" + shard.getInclusiveBeginKey());
                System.out.println("Shard end key is :" + shard.getExclusiveEndKey());
            }
            System.out.println("ready to split shard");
            // Split Shard 0. 
            client.SplitShard(projectName, logstoreName, 0, "3f000000000000000000000000000000");

            // Wait for 60 seconds before you query shards again. 
            TimeUnit.SECONDS.sleep(60);

            System.out.println(String.format("split shards of %s success", logstoreName));

            // Query all shards after splitting. 
            ListShardResponse response1 = client.ListShard(projectName, logstoreName);
            for(Shard shard1 : response1.GetShards()){
                System.out.println("Shard ID is :" + shard1.getShardId());
                System.out.println("Shard status is :" + shard1.getStatus());
                System.out.println("Shard begin key is :" + shard1.getInclusiveBeginKey());
                System.out.println("Shard end key is :" + shard1.getExclusiveEndKey());
            }
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

Expected result:

ready to list shard
Shard ID is :0
Shard status is :readwrite
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard ID is :1
Shard status is :readwrite
Shard begin key is :7f000000000000000000000000000000
Shard end key is :ffffffffffffffffffffffffffffffff
ready to split shard
split shards of ali-test-logstore success
Shard ID is :2
Shard status is :readwrite
Shard begin key is :00000000000000000000000000000000
Shard end key is :3f000000000000000000000000000000
Shard ID is :3
Shard status is :readwrite
Shard begin key is :3f000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard ID is :1
Shard status is :readwrite
Shard begin key is :7f000000000000000000000000000000
Shard end key is :ffffffffffffffffffffffffffffffff
Shard ID is :0
Shard status is :readonly
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000

Sample code that is used to merge shards

The following sample code provides an example on how to merge the shards of a Logstore:

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Shard;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.ListShardResponse;

import java.util.concurrent.TimeUnit;

public class MergeShard {
    public static void main(String[] args) throws LogException {
        // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // The name of the project. 
        String projectName = "ali-test-project";
        // The name of the Logstore. 
        String logstoreName = "ali-test-logstore";
        // The Simple Log Service endpoint. In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. Replace the parameter value with the actual endpoint. 
        String host = "https://cn-hangzhou.log.aliyuncs.com";

        // Create a Simple Log Service client. 
        Client client = new Client(host, accessId, accessKey);

        try {
            System.out.println("ready to list shard");

            // Query all shards before merging. 
            ListShardResponse response = client.ListShard(projectName, logstoreName);
            for(Shard shard : response.GetShards()){
                System.out.println("Shard ID is :" + shard.getShardId());
                System.out.println("Shard status is :" + shard.getStatus());
                System.out.println("Shard begin key is :" + shard.getInclusiveBeginKey());
                System.out.println("Shard end key is :" + shard.getExclusiveEndKey());
            }
            System.out.println("ready to merge shard");
            // Merge Shard 2 and Shard 3. Simple Log Service automatically identifies the neighbor shard for merging. 
            client.MergeShards(projectName, logstoreName, 2);

            // Wait for 60 seconds before you query shards again. 
            TimeUnit.SECONDS.sleep(60);

            System.out.println(String.format("merge shards of %s success", logstoreName));

            // Query all shards after merging. 
            ListShardResponse response1 = client.ListShard(projectName, logstoreName);
            for(Shard shard1 : response1.GetShards()){
                System.out.println("Shard ID is :" + shard1.getShardId());
                System.out.println("Shard status is :" + shard1.getStatus());
                System.out.println("Shard begin key is :" + shard1.getInclusiveBeginKey());
                System.out.println("Shard end key is :" + shard1.getExclusiveEndKey());
            }
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

Expected result:

ready to list shard
Shard ID is :2
Shard status is :readwrite
Shard begin key is :00000000000000000000000000000000
Shard end key is :3f000000000000000000000000000000
Shard ID is :3
Shard status is :readwrite
Shard begin key is :3f000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard ID is :1
Shard status is :readwrite
Shard begin key is :7f000000000000000000000000000000
Shard end key is :ffffffffffffffffffffffffffffffff
Shard ID is :0
Shard status is :readonly
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
ready to merge shard
merge shards of ali-test-logstore success
Shard ID is :4
Shard status is :readwrite
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard ID is :1
Shard status is :readwrite
Shard begin key is :7f000000000000000000000000000000
Shard end key is :ffffffffffffffffffffffffffffffff
Shard ID is :0
Shard status is :readonly
Shard begin key is :00000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000
Shard ID is :2
Shard status is :readonly
Shard begin key is :00000000000000000000000000000000
Shard end key is :3f000000000000000000000000000000
Shard ID is :3
Shard status is :readonly
Shard begin key is :3f000000000000000000000000000000
Shard end key is :7f000000000000000000000000000000

References

If the response that is returned by Log Service contains error information after you call an API operation, the call fails. You can handle errors based on the error codes that are returned when API calls fail. For more information, see Error codes.
Alibaba Cloud OpenAPI Explorer provides debugging capabilities, SDKs, examples, and related documents. You can use OpenAPI Explorer to debug Log Service API operations without the need to manually encapsulate or sign requests. For more information, visit OpenAPI Portal.
Log Service provides the command-line interface (CLI) to meet the requirements for automated configurations in Log Service. For more information, see Log Service CLI.
For more information about shard-related operations, see the following topics:
For more information about sample code, see Alibaba Cloud Log Service SDK for Java on GitHub.