All Products
Search
Document Center

Simple Log Service:Consume log data

Last Updated:Jan 25, 2024

Simple Log Service provides SDKs for multiple programming languages. You can use the SDKs to consume log data. This topic provides an example on how to use an SDK to consume log data. This topic also describes the consumption preview feature that you can use to consume data in the Simple Log Service console.

Example of SDK-based consumption

The following example shows how to use Simple Log Service SDK for Java to consume log data after you create a PullLogsDemo.java file. Sample command: For more information, see Overview of Simple Log Service SDK.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Consts;
import com.aliyun.openservices.log.common.LogGroupData;
import com.aliyun.openservices.log.common.Shard;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.PullLogsRequest;
import com.aliyun.openservices.log.response.ListShardResponse;
import com.aliyun.openservices.log.response.PullLogsResponse;

import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class PullLogsDemo {
    // The Simple Log Service endpoint. // In this example, the Simple Log Service endpoint for the China (Hangzhou) region is used. You must replace the value with an actual endpoint based on your business requirements.
    private static final String endpoint = "cn-hangzhou.log.aliyuncs.com";
    // In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
    private static final String accessKeyId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
    private static final String accessKeySecret = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
    // The project name.
    private static final String project = "your_project";
    // The Logstore name.
    private static final String logStore = "your_logstore";

    public static void main(String[] args) throws Exception {
        // Create a client for Simple Log Service.
        Client client = new Client(endpoint, accessKeyId, accessKeySecret);
        // Query the shards of the Logstore.
        ListShardResponse resp = client.ListShard(project, logStore);
        System.out.printf("%s has %d shards\n", logStore, resp.GetShards().size());
        Map<Integer, String> cursorMap = new HashMap<Integer, String>();
        for (Shard shard : resp.GetShards()) {
            int shardId = shard.getShardId();
            // Use the BEGIN cursor or obtain a specific cursor to consume log data. If you want to use the END cursor to consume log data, use Consts.CursorMode.END.
            cursorMap.put(shardId, client.GetCursor(project, logStore, shardId, Consts.CursorMode.BEGIN).GetCursor());
        }
        try {
            while (true) {
                // Obtain log data from each shard.
                for (Shard shard : resp.GetShards()) {
                    int shardId = shard.getShardId();
                    PullLogsRequest request = new PullLogsRequest(project, logStore, shardId, 1000, cursorMap.get(shardId));
                    PullLogsResponse response = client.pullLogs(request);
                    // Split log groups into logs by logic. 
                    List<LogGroupData> logGroups = response.getLogGroups();
                    System.out.printf("Get %d logGroup from logStore:%s:\tShard:%d\n", logGroups.size(), logStore, shardId);
                    // Move the cursor after the pulled logs are processed. 
                    cursorMap.put(shardId, response.getNextCursor());
                }
            }
        } catch (LogException e) {
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Consumption preview

You can use the consumption preview feature to consume log data. The consumption preview feature allows you to preview specific log data that is stored in a Logstore in the Simple Log Service console.

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the Logstore that you want to manage.

  4. Find the Logstore from which you want to consume log data and choose 日志预览 > Consumption Preview.

  5. In the Consumption Preview panel, select a shard and a time range. Then, click Preview.

    The log data of the first 10 packets in the specified time range is displayed in the Consumption Preview panel.xiaofei