GetLogs

Last Updated: Apr 19, 2017

On the GetHistograms interface, queries the log data in a LogStore under a specified project. By specifying the relevant parameters, it can also just query log data matching the specified criteria.

When a log is written in a Logstore, the delay at which log query interfaces (GetHistograms and GetLogs) can query this log varies with the log type. Based on a log’s time stamp, the log service classifies it as one of two types:

  • Real-time data: The time point for a log is the current time point for the sever (-180 seconds, 900 seconds]. For example, if the log time is UTC 2014-09-25 12:03:00 and the time point at which the server receives the log is UTC 2014-09-25 12:05:00, the log is used as the real-time data, which usually appears in normal scenarios.
  • Historical data: The time point for a log is the current time point for the sever (-7 x 86400 seconds, -180 seconds]. For example, if the log time is UTC 2014-09-25 12:00:00 and the time point at which the server receives the log is UTC 2014-09-25 12:05:00, the log is used as the real-time data, which usually appears in scenarios of supplementation.

The maximum delay between real-time data writing and query is 3 seconds. (99.9% of the time the data can be queried within 1 second).

Request syntax

  1. GET /logstores/<logstorename>?type=histogram&topic=<logtopic>&from=<starttime>&to=<endtime>&query=<querystring>&line=<linenum>&offset=<startindex>&reverse=<ture|false> HTTP/1.1
  2. Authorization: <AuthorizationString>
  3. Date: <GMT Date>
  4. Host: <Project Endpoint>
  5. x-log-bodyrawsize: 0
  6. x-log-apiversion: 0.6.0
  7. x-log-signaturemethod: hmac-sha1

Request parameters

Name Type Required Description
logstorename String Yes The LogStore name of the logs to query.
type String Yes The type of the LogStore data to query. On the GetLogs interface, this parameter must be “log”.
from Integer Yes The query start time (to the second, from 1970-1-1 00:00:00 UTC).
to Integer Yes The query end time (to the second, from 1970-1-1 00:00:00 UTC).
topic String No The log topic to query.
query String No Query expression. For details about the query expression syntax, refer to Query Syntax.
line Integer No The maximum number of log lines returned for the request. The value ranges from 0 to 100. The default value is 100.
offset Integer No The returned log start point for the request. The value can be 0 or a positive integer. The default value is 0.
reverse Boolean No Whether or not log are returned in reverse order according to the log time stamp. “true” indicates reverse order and “false” indicates regular order. The default value is “false”.

Request header

The GetLogs interface does not have a special request header. For details about the public request header of the log service API, refer to Public Request Header.

Response header

The GetLogs interface does not have a special response header. For details about the public response header of the log service API, refer to Public Response Header.

Response element

When a GetLogs request is successful, the response body will contain log data hit in the query. When you need to query an extremely large amount of log data, the response results from this interface may be incomplete. Therefore, the response element contains a specialized member that indicates whether or not the request’s return results are complete. The specific response element format is as follows:

Name Type Description
progress String The query results status. The two values, Incomplete and Complete, indicate whether or not the results are complete.
count Integer The total number of logs in the current query results.
logs Array The original log data for the current query results. The specific structure is described below.

The structure of each element in the logs array is shown below:

Name Type Description
__time__ Integer The log time stamp (to the second, from 1970-1-1 00:00:00 UTC).
__source__ String The log source, specified when logs are written.
[content] Key-Value pair Original log content, organized in Key-value pair form.

Detailed description

  • All the time ranges used in this interface follow the “Left closed and right open” principle. That is, these time ranges include the start time, but not the end time. If the from and to values are the same, the time range is invalid and the function will return an error.
  • As described above, each call to this interface must return results within a limited time range, and thus each query can only scan a specified number of logs. If a request requires that an extremely large amount of data be processed, the return results for the request may be incomplete (the progress member in the return results indicates whether or not they are complete). At the same time, the server will cache query results within 15 minutes. When a portion of the query request results are cache hits, the server will continue to scan the log data that are not cache hits for this request. To reduce the workload of merging multiple query results, the server cache hit query results and the new hits of the current query are merged and returned to you. Therefore, the log service allows you to call the interface multiple times with the same parameters to finally obtain the complete results. Because the log data volume involved in the query changes massively, the log service API cannot predict how many calls are required to obtain complete results from the interface. Therefore, you must check the progress member value in the return results of each request to determine if they need to continue. You must note that each duplicate call to this interface will consume the same number of query CUs again.

Specific error codes

In addition to general error codes of the log service API, the GetLogs interface may return the following special error codes:

HTTP Status Code Error Code Error Message Description
404 LogStoreNotExist logstore {Name} not exist. The LogStore does not exist.
400 InvalidTimeRange request time range is invalid. The request time range is invalid.
400 InvalidQueryString query string is invalided The request’s query string is invalid.
400 InvalidOffset offset is invalided The request’s offset parameter is invalid.
400 InvalidLine line is invalided The request’s line parameter is invalid.
400 InvalidReverse Reverse value is invalid The Reverse parameter value is invalid.

The {name} variable in the above error messages will be replaced by the actual logstore name.

Example

Take a project named big-game in Hangzhou as an example. Query the logs themed groupA in the LogStore named app_log. The query ranges from 2014-09-01 00:00:00 to 2014-09-01 22:00:00, and the keyword is “error”. The query starts from the range header, and a maximum of 20 logs are returned.

Request example:
  1. GET /logstores/app_logtype=log&topic=groupA&from=1409529600&to=1409608800&query=error&line=20&offset=0 HTTP/1.1
  2. Authorization: <AuthorizationString>
  3. Date: Wed, 3 Sept. 2014 08:33:46 GMT
  4. Host: big-game.cn-hangzhou.log.aliyuncs.com
  5. x-log-bodyrawsize: 0
  6. x-log-apiversion: 0.4.0
  7. x-log-signaturemethod: hmac-sha1
Response example:
  1. HTTP/1.1 200 OK
  2. Content-MD5: 36F9F7F0339BEAF571581AF1B0AAAFB5
  3. Content-Type: application/json
  4. Content-Length: 269
  5. Date: Wed, 3 Sept. 2014 08:33:47 GMT
  6. x-log-requestid: efag01234-12341-15432f
  7. {
  8. "progress": "Complete",
  9. "count": 2,
  10. "logs": [
  11. {
  12. "__time__": 1409529660,
  13. "__source__": "10.237.0.17",
  14. "Key1": "error",
  15. "Key2": "Value2"
  16. },
  17. {
  18. "__time__": 1409529680,
  19. "__source__": "10.237.0.18",
  20. "Key3": "error",
  21. "Key4": "Value4"
  22. }
  23. ]
  24. }

In this response example, the progress member status is Complete. This indicates that the entire LogStore has been queried and the return results are complete. For this request, a total of two logs meet the query criteria and the log data is in the “logs” member. If, in the response results, the progress member has a status of Incomplete, you need to repeat the request to obtain the complete results.

Thank you! We've received your feedback.