GetLogs API examples - Query logs with Java SDK - Simple Log Service

Call the GetLogs API to query collected logs using the SLS Java SDK.

Prerequisites

Simple Log Service is activated. For more information, see Activate Simple Log Service.
Simple Log Service SDK for Java is initialized. For more information, see Initialize Simple Log Service SDK for Java.

Usage notes

In this example, the public Simple Log Service endpoint for the China (Hangzhou) region is used. Endpoint: https://cn-hangzhou.log.aliyuncs.com.
If you want to access Simple Log Service from other Alibaba Cloud services that reside in the same region as your project, you can use the internal Simple Log Service endpoint, which is https://cn-hangzhou-intranet.log.aliyuncs.com.
For more information about the supported regions and endpoints of Simple Log Service, see Endpoints.
Call IsCompleted() on the SDK response object to check whether the result set is complete.
- true: The query succeeded and the result set is complete.
- false: The query succeeded but the result set is incomplete. Repeat the request to obtain full results. Possible causes of inaccurate queries.

Parameters

Request parameters

Name	Type	Required	Description
project	String	Yes	The name of the project. The project in Simple Log Service is used to isolate the resources of different users and control access to specific resources. See Manage projects.
logstore	String	Yes	The name of the logstore. The logstore in Simple Log Service is used to collect, store, and query logs. See Manage a logstore.
from	int	Yes	The beginning of the time range to query. The value is a UNIX timestamp. Note The time when the logstore receives logs. The `__tag__:__receive_time__` field is a reserved field of Simple Log Service. The time range defined by the start time and the end time. The time range is a left-closed, right-open interval. The interval includes the start time of the time range, but not the end time. If both are the same, the interval is invalid and an error is returned. To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, the time range is used for query and analysis. If you want to specify a time range that is accurate to the second, you can use date and time functions to convert the time format. Examples: `* \| SELECT * FROM log WHERE from_unixtime(__time__) > from_unixtime(1664186624) AND from_unixtime(__time__) < now()` `* \| SELECT * FROM log WHERE __time__ > to_unixtime(date_parse('2022-10-19 15:46:05', '%Y-%m-%d %H:%i:%s')) AND __time__ < to_unixtime(now())`
to	int	Yes	The end of the time range to query. The value is a UNIX timestamp. Note The time when the logstore receives logs. The `__tag__:__receive_time__` field is a reserved field of Simple Log Service. The time range defined by the start time and the end time. The time range is a left-closed, right-open interval. The interval includes the start time of the time range, but not the end time. If both are the same, the interval is invalid and an error is returned. To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, the time range is used for query and analysis. If you want to specify a time range that is accurate to the second, you can use date and time functions to convert the time format. Examples: `* \| SELECT * FROM log WHERE from_unixtime(__time__) > from_unixtime(1664186624) AND from_unixtime(__time__) < now()` `* \| SELECT * FROM log WHERE __time__ > to_unixtime(date_parse('2022-10-19 15:46:05', '%Y-%m-%d %H:%i:%s')) AND __time__ < to_unixtime(now())`
topic	String	No	The topic of logs. The default value is an empty string. For more information, see Log topics.
query	String	No	A search or analytic statement. Query and Analysis Overview. To use SQL Exclusive, add `set session parallel_sql=true;` to the analytic statement. Example: `* \| set session parallel_sql=true; select count() as pv`. Common errors in querying and analyzing logs. Note* If query contains an analytic statement (SQL), the line and offset parameters are invalid. Set them to 0 and use the LIMIT clause for paging. Paginate query and analysis results.
line	int	No	Valid only for search statements. Maximum number of logs to return. Range: 0 to 100. Default: 100.
offset	int	No	Valid only for search statements. Starting row for the query. Default: 0.
reverse	boolean	No	Whether to return logs in descending timestamp order. Precision: minute-level. true: Descending order (newest first). false (default): Ascending order (oldest first). Important reverse is valid only for search statements. It controls the sorting order of returned logs. For query and analysis statements, reverse is invalid. Use the `order by` clause in SQL instead.
powerSql	boolean	No	Whether to use Dedicated SQL. High-performance and accurate query and analysis (Dedicated SQL). true: Use Dedicated SQL. false (default): Use Standard SQL. You can also enable Dedicated SQL by adding the configuration to the query parameter.
scan	boolean	No	Whether to use scan mode for the query.
forward	boolean	No	For scan queries only. true: retrieves the next page. false: retrieves the previous page.

Response parameters

GetLogs.

Raw log

body_bytes_sent:1750
host:www.example.com
http_referer:www.example.com
http_user_agent:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; it-it) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
http_x_forwarded_for:203.0.XX.XX
remote_addr:203.0.XX.XX
remote_user:p288
request_length:13741
request_method:GET
request_time:71
request_uri:/request/path-1/file-1
http_code:200
time_local:11/Aug/2021:06:52:27
upstream_response_time:0.66

Query and analysis examples

The following Java examples demonstrate common query patterns.

Important

When calling GetLogs with the SLS Java SDK:

If query contains only a search statement (e.g., request_method:POST), the line parameter controls the number of returned logs (max 100). To return more than 100 logs, use the LIMIT clause in SQL. LIMIT clause.
If query contains a query and analysis statement (e.g., request_method:POST | SELECT host, COUNT(*) AS pv GROUP BY host LIMIT 5), the line parameter is ignored. Use the LIMIT clause in SQL to control the number of rows. LIMIT clause.

Basic syntax.

Example 1: Query logs using a keyword

Query logs by keyword path-0/file-5. The line parameter limits results to 3 logs.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Use the keyword "path-0/file-5" to query logs.
            String query = "path-0/file-5";

            int from = 1754449503;
            int to = 1754449510;

            // In this example, the query parameter is used to set the query statement. The line parameter is used to control the number of returned logs. The value is 3, and the maximum value is 100.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Response:

-------------Query is started.-------------
Returned query result count :3
from time is :1644573549
to time is :1644573849
log time : 1644573808
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","request_uri":"/request/path-0/file-5"...}
log time : 1644573808
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","request_uri":"/request/path-0/file-5"...}
log time : 1644573788
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","request_uri":"/request/path-0/file-5"...}
-------------Query is finished.-------------

Process finished with exit code 0

Example 2: Query logs by specifying a field

Query logs where the request method is POST. The line parameter limits results to 3 logs.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Query for logs where the request method is POST.
            String query = "request_method:POST";
            // Set the time range for the query.
            int from = 1754449503;
            int to = 1754449510;

            // In this example, the query parameter is used to set the query statement. The line parameter is used to control the number of returned logs. The value is 3, and the maximum value is 100.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Response:

-------------Query is started.-------------
Returned query result count :3
from time is :1644574151
to time is :1644574451
log time : 1644574438
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","body_bytes_sent":"3604","request_method":"POST"...}
log time : 1644574438
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","body_bytes_sent":"3369","request_method":"POST"...}
log time : 1644574438
Jsonstring : {"remote_addr":"203.0.XX.XX","__topic__":"nginx_access_log","body_bytes_sent":"12714","request_method":"POST"...}
-------------Query is finished.-------------

Process finished with exit code 0

Example 3: Analyze logs using an SQL statement

Query logs where the request method is POST and count the page views (PVs).

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Query for logs where the request method is POST and count the number of PVs for the POST requests.
            String query = "request_method:POST|select COUNT(*) as pv";
            // Set the time range for the query.
            int from = 1754449503;
            int to = 1754449510;

            // In this example, the query parameter is set to a query and analysis statement. The line parameter is invalid.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Response:

-------------Query is started.-------------
Returned query result count :1
from time is :1644574354
to time is :1644574654
log time : 1644574354
Jsonstring : {"pv":"162","logtime":1644574354}
-------------Query is finished.-------------

Process finished with exit code 0

Example 4: Analyze logs using the GROUP BY clause

Query logs where the request method is POST and group results by host.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Query for logs where the request method is POST and group the results by host.
            // Use the LIMIT clause in the SQL syntax to limit the number of returned rows to 5.
            String query = "request_method:POST|select host, COUNT(*) as pv group by host limit 5";
            // Set the time range for the query.
            int from = 1754449503;
            int to = 1754449510;

            // In this example, the query parameter is set to a query and analysis statement. The line parameter is invalid. The number of rows to return is determined by the LIMIT clause in the query.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Response:

-------------Query is started.-------------
Returned query result count :5
from time is :1644574445
to time is :1644574745
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example1.com","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.org","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.net","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.edu","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.aliyundoc.com","logtime":1644574445}
-------------Query is finished.-------------

Process finished with exit code 0

Example 5: Analyze logs using the GROUP BY clause (200 logs returned)

Query logs where the request method is POST, group by host, and return up to 200 rows using the LIMIT clause.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Query for logs where the request method is POST and group the results by host.
            // Use the LIMIT clause in the SQL syntax to control the number of rows to return.
            String query = "request_method:POST|select host, COUNT(*) as pv group by host limit 0,200";
            // Set the time range for the query.
            int from = 1754449503;
            int to = 1754449510;

            // In this example, the query parameter is set to a query and analysis statement. The line parameter is invalid. The number of rows to return is determined by the LIMIT clause in the query.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Response:

-------------Query is started.-------------
Returned query result count :200
from time is :1644574445
to time is :1644574745
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example1.com","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.org","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.net","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.example.edu","logtime":1644574445}
log time : 1644574445
Jsonstring : {"pv":"1","host":"www.aliyundoc.com","logtime":1644574445}
......
-------------Query is finished.-------------

Process finished with exit code 0

Example 6: Query the total number of logs within the previous hour using an SQL statement

Use *|select count(*) as count to get the total number of logs from the last hour.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.LogItem;
import com.aliyun.openservices.log.common.QueriedLog;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.request.GetLogsRequest;
import com.aliyun.openservices.log.response.GetLogsResponse;

import java.util.Date;

public class GetLogsTest {

    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Enter the project name.
        String project = "your-project-name";
        // Set the endpoint for Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "cn-hangzhou.log.aliyuncs.com";
        // Enter the Logstore name.
        String logStore = "your-logstore-name";

        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);

        // Execute a query in the specified Logstore.
        try {
            // Query the total number of logs.
            String query = "*|select count(*) as count";
            // Set the query time range to 1 hour (3,600 seconds).
            int from = (int) (new Date().getTime() / 1000 - 3600);
            int to = (int) (new Date().getTime() / 1000);
            int offset = 0;
            int line = 200;
            // In this example, the SQL statement in the query parameter is used to query the total number of logs in the specified time range. The line and offset parameters are invalid for SQL queries.
            GetLogsRequest request = new GetLogsRequest(project, logStore, from, to, "", query,line,offset,true);
            GetLogsResponse logsResponse = client.GetLogs(request);
            System.out.println("-------------Query is started.-------------");
            System.out.println("Returned query result count :" + logsResponse.GetCount());
            System.out.println("from time is :" + from);
            System.out.println("to time is :" + to);
            for (QueriedLog log : logsResponse.getLogs()) {
                LogItem item = log.GetLogItem();
                System.out.println("log time : " + item.mLogTime);
                System.out.println("Jsonstring : " + item.ToJsonString());
            }
            System.out.println("-------------Query is finished.-------------");

        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

The total number of logs in the previous hour is 19,051. Response:

from time is :1675041679
to time is :1675045279
Returned sql result count :1
Jsonstring : {"count":"19051","logtime":1675041679}
-------------Query is finished.-------------

References

Error codes.
In addition to its native SDK, SLS also supports the common Alibaba Cloud SDKs. For more information, see Simple Log Service_SDK Center_Alibaba Cloud OpenAPI Explorer.
Simple Log Service CLI.
Alibaba Cloud Simple Log Service SDK for Java on GitHub.