Queries logs that are generated within a specified time range from a specified Logstore of a specified project.
Usage notes
Simple Log Service allows you to create a Scheduled SQL job. For more information, see Create a Scheduled SQL job.
Host consists of a project name and a Simple Log Service endpoint. You must specify a project in Host.
An AccessKey pair is created and obtained. For more information, see AccessKey pair.
The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in Simple Log Service is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. Make sure that the RAM user has the management permissions on Simple Log Service resources. For more information, see Create a RAM user and authorize the RAM user to access Simple Log Service.
The information that is required to query logs is obtained. The information includes the name of the project to which the logs belong, the region of the project, and the name of the Logstore to which the logs belong. For more information, see Manage a project and Manage a Logstore.
Limits are imposed when you use Simple Log Service to query logs. We recommend that you specify query statements and query time ranges based on the limits. For more information, see Log search overview and Log analysis overview.
Indexes are configured before you query logs. For more information, see Create indexes.
If the number of logs in a Logstore significantly changes, Simple Log Service cannot predict the number of times that you must call this operation to obtain the complete results. In this case, you must check the value of the x-log-progress parameter in the response of each request and determine whether to call this operation one more time to obtain the complete results. Each time you call this operation, the same number of charge units (CUs) are consumed.
After a log is written to a Logstore, you can call the GetHistograms or the GetLogs operation to query the log. The latency of the query varies based on the type of the log. Simple Log Service classifies logs into the following types based on log timestamps:
Real-time data: The difference between the time record in a log of this type and the current time on Simple Log Service is within the interval (-180 seconds,900 seconds]. For example, if a log was generated at 12:03:00, September 25, 2014 (UTC) and Simple Log Service received the log at 12:05:00, September 25, 2014 (UTC), Simple Log Service processes the log as real-time data. This type of log is usually generated in common scenarios.
Historical data: The difference between the time record in a log of this type and the current time on Simple Log Service is within the interval [-604,800 seconds,-180 seconds). For example, if a log was generated at 12:00:00, September 25, 2014 (UTC) and Simple Log Service received the log at 12:05:00, September 25, 2014 (UTC), Simple Log Service processes the log as historical data. This type of log is usually generated in data backfill scenarios.
After real-time data is written to a Logstore, the data can be queried with a maximum latency of 3 seconds. For 99.9% of queries, the latency is no more than 1 second.
Simple Log Service calculates the difference between the log time that is specified by the __time__ field and the receiving time that is specified by the __tag__:__receive_time__ field for each log. The receiving time indicates when Simple Log Service receives the log. If the difference is within the interval (-180 seconds,900 seconds], Simple Log Service processes the log as real-time data. If the difference is within the interval [-604,800 seconds,-180 seconds), Simple Log Service processes the log as historical data.
Simple Log Service provides examples on how to call the GetLogs operation by using Simple Log Service SDK for Java and Simple Log Service SDK for Python. For more information, see Examples of calling the GetLogs operation by using Simple Log Service SDK for Java and Examples of calling the GetLogs operation by using Simple Log Service SDK for Python.
Authentication resources
The following table describes the authorization information that is required for this operation. You can add the information to the Action element of a RAM policy statement to grant a RAM user or a RAM role the permissions to call this operation.
Action | Resource |
|
|
Debugging
Request headers
This operation uses only common request headers. For more information, see Common request headers.
Request syntax
GET /logstores/{logstore}?type=log HTTP/1.1
Request parameters
Parameter | Type | Position | Required | Example | Description |
project | String | Host | Yes | ali-test-project | The name of the project. |
logstore | String | Path | Yes | example-logstore | The Logstore whose data you want to query. |
from | Integer | Query | Yes | 1627268185 | The beginning of the time range to query. The value is the log time that is specified when log data is written.
Note To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, the time range is used for query and analysis. If you want to specify a time range that is accurate to the second, you must use the from_unixtime or to_unixtime function in your analytic statement to convert the time format. For more information about the functions, see from_unixtime function and to_unixtime function. Examples:
|
to | Integer | Query | Yes | 1627269085 | The end of the time range to query. The value is the log time that is specified when log data is written.
Note To ensure that full data can be queried, specify a query time range that is accurate to the minute. If you also specify a time range in an analytic statement, the time range is used for query and analysis. If you want to specify a time range that is accurate to the second, you must use the from_unixtime or to_unixtime function in your analytic statement to convert the time format. For more information about the functions, see from_unixtime function and to_unixtime function. Examples:
|
query | String | Query | No | status: 401 | SELECT remote_addr,COUNT(*) as pv GROUP by remote_addr ORDER by pv desc limit 5 | The search statement or the query statement. For more information, see Log search overview and Log analysis overview. If you add Note If you specify an analytic statement in the query parameter, the line and offset parameters do not take effect in this operation. In this case, we recommend that you set the line and offset parameters to 0 and use the LIMIT clause to limit the number of logs to return on each page. For more information, see Perform paged queries. |
topic | String | Query | No | topic | The topic of the logs. The default value is an empty string. For more information, see Topic. |
line | Long | Query | No | 100 | The maximum number of logs to return for the request. This parameter takes effect only when the query parameter is set to a search statement. Minimum value: 0. Maximum value: 100. Default value: 100. For more information, see Perform paged queries. |
offset | Long | Query | No | 0 | The row from which the query starts. This parameter takes effect only when the query parameter is set to a search statement. Default value: 0. Default value: 0. For more information, see Perform paged queries. |
reverse | Boolean | Query | No | false | Specifies whether to return logs in reverse chronological order of log timestamps. The log timestamps are accurate to the minute. Valid values:
Important
|
powerSql | Boolean | Query | No | false | Specifies whether to use Dedicated SQL. For more information, see Enable Dedicated SQL. Valid values:
You can use the powerSql or query parameter to configure Dedicated SQL. |
Response parameters
Parameter | Type | Example | Description |
x-log-progress | String | Complete | The status of the query and analysis results that are returned. Valid values:
Note You can use the |
x-log-count | Long | 100 | The number of rows that are returned. Note The total number of logs that meet the specified conditions is not returned. To query the total number of logs that meet the specified conditions, you must specify a query statement. For example, the query statement |
x-log-processed-rows | Long | 10000 | The number of lines that are processed based on the request. |
x-log-elapsed-millisecond | Long | 5 | The time that is consumed by the request. Unit: milliseconds. |
Server | String | nginx | The name of the server. |
Content-Type | String | application/json | The content type of the response body. |
Content-Length | String | 0 | The content length of the response body. |
Connection | String | close | Indicates whether the connection is persistent. Valid values:
|
Date | String | Sun, 27 May 2018 08:25:04 GMT | The time when the response was returned. |
x-log-requestid | String | 5B0A6B60BB6EE39764D458B5 | The unique ID of the request. The ID is generated by the server. |
Array of Object | [{'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.100.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}] | An array of logs. Each element in the array indicates a log. |
Examples
Sample requests
GET /logstores/{logstore}?type=log&from=1627268185&to=1627269085&query=status: 401 | SELECT remote_addr,COUNT(*) as pv GROUP by remote_addr ORDER by pv desc limit 5&topic=topic&line=100&offset=0&reverse=false&powerSql=false HTTP/1.1
Host:ali-test-project.cn-hangzhou.log.aliyuncs.com
Content-Type:application/json
Description of the sample requests
### Sample raw log
```json
{
"remote_addr": "203.0.*.*",
"__time__": 1649904328,
"__topic__": "nginx_access_log",
"__source__": "127.0.0.1",
"body_bytes_sent": "3841",
"__tag__:__receive_time__": "1649904334",
"time_local": "14/Apr/2022:02:45:28",
"request_method": "HEAD",
"request_uri": "/request/path-2/file-2",
"http_user_agent": "Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:21.0.0) Gecko/20121011 Firefox/21.0.0",
"remote_user": "jlqd8",
"request_time": "17",
"__pack_meta__": "1|MTYyODY2Mjk0NzQ1ODU1Mjk4Mg==|197|165",
"request_length": "4184",
"http_referer": "www.example.com",
"__tag__:__receive_time___0": "1649904334",
"host": "www.example.com",
"http_x_forwarded_for": "198.51.100.48",
"upstream_response_time": "0.23",
"status": "401"
}
```
Sample success responses
JSON
format
HTTP/1.1 200 OK
Content-Type:application/json
{
"x-log-progress" : "Complete",
"x-log-count" : 5,
"logs" : "[{'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.XXX.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}, {'remote_addr': '198.51.100.XXX', 'pv': '1', '__source__': '', '__time__': '1649902984'}]"
}
Error codes
For a list of error codes, see Service error codes.
HTTP status code | Error code | Error message | Description |
404 | ProjectNotExist | Project does not exist. | The specified project does not exist. |
404 | LogStoreNotExist | Logstore does not exist. | The specified Logstore does not exist. |
400 | InvalidTimeRange | Request time range is invalid. | The specified time range is invalid. |
400 | InvalidQueryString | Query string is invalid. | The specified query statement is invalid. |
400 | InvalidOffset | Offset is invalid. | The specified offset parameter is invalid. |
400 | InvalidLine | Line is invalid. | The specified line parameter is invalid. |
400 | InvalidReverse | Reverse value is invalid. | The specified reverse parameter is invalid. |
400 | IndexConfigNotExist | Logstore without index config. | The indexing feature is not enabled for the Logstore. |
400 | ParameterInvalid | ErrorType:OLSQueryParseError.ErrorMessage:offset is not available for pagination in sql query, please use limit x,y syntax for pagination. | The specified analytic statement in the query parameter of the request is invalid. If the query parameter is set to a query statement that contains an analytic statement, we recommend that you set the line and offset parameters to 0 and specify a LIMIT clause for pagination. For more information about how to troubleshoot the errors that are related to the SQL statement specified by the query parameter, see How do I resolve common errors that may occur when I query and analyze logs? |
500 | InternalServerError | Specified Server Error Message. | An internal server error has occurred. |
For more information, see Common error codes.