GetHistograms

Last Updated: Mar 31, 2017

On the GetHistograms interface, queries the log distribution in a Logstore under a specified project. By specifying the relevant parameters, it can also just query log distributions matching the specified criteria.

When a log is written in a Logstore, the delay at which log query interfaces (GetHistograms and GetLogs) can query this log varies with the log type. Based on a log’s time stamp, the log service classifies it as one of two types:

  • Real-time data: The time point for a log is the current time point for the sever (-180 seconds, 900 seconds]. For example, if the log time is UTC 2014-09-25 12:03:00 and the time point at which the server receives the log is UTC 2014-09-25 12:05:00, the log is used as the real-time data, which usually appears in normal scenarios.
  • Historical data: The time point for a log is the current time point for the sever (-7 x 86400 seconds, -180 seconds]. For example, if the log time is UTC 2014-09-25 12:00:00 and the time point at which the server receives the log is UTC 2014-09-25 12:05:00, the log is used as the real-time data, which usually appears in scenarios of supplementation.

The delay between real-time data writing and query is 1 minute.

Request syntax

  1. GET /logstores/<logstorename>?type=histogram&topic=<logtopic>&from=<starttime>&to=<endtime>&query=<querystring> HTTP/1.1
  2. Authorization: <AuthorizationString>
  3. Date: <GMT Date>
  4. Host: <Project Endpoint>
  5. x-log-bodyrawsize: 0
  6. x-log-apiversion: 0.6.0
  7. x-log-signaturemethod: hmac-sha1

Request parameters

Name Type Required Description
logstorename String Yes The LogStore name of the logs to query.
type String Yes The type of LogStore data to query. In the GetHistograms interface, this parameter must be set to “histogram”.
from Integer Yes The query start time (to the second, from 1970-1-1 00:00:00 UTC).
to Integer Yes The query end time (to the second, from 1970-1-1 00:00:00 UTC).
topic String No The log topic to query.
query String No Query expression. For details about the query expression syntax, refer to Query Syntax.

Request header

The GetHistograms interface does not have a special request header. For details about the public request header of the log service API, refer to Public Request Header.

Response header

The GetHistograms interface does not have a special response header. For details about the public response header of the log service API, refer to Public Response Header.

Response element

When a GetHistograms request is successful, the response body will contain the distribution of the number of log hits returned by the query along the time axis. The response results will evenly divide the time range queried by you into several (1 to 60) subintervals and return the number of log hits for each subinterval. Because the SLS must return results within a limited time range to ensure real-time functionality, each query can only scan a specified number of logs. When you need to query an extremely large amount of log data, the response results from this interface may be incomplete. Therefore, the response element contains a specialized member that indicates whether or not the request’s return results are complete. The specific response element format is as follows:

Name Type Description
progress String The query results status. The two values, Incomplete and Complete, indicate whether or not the results are complete.
count Integer The total of all logs in the current query results.
histograms Array The distribution of the current query results among the subintervals. For the actual structure, see the description below.

The structure of each element in the histograms array is shown below:

Name Type Description
from Integer The start time for subintervals (to the second, from 1970-1-1 00:00:00 UTC
to Integer The end time for subintervals (to the second, from 1970-1-1 00:00:00 UTC)
count Integer The number of log hits for this subinterval in the query results
progress String Indicates whether or not the query results for this subinterval are complete. Values: Incomplete and Complete.

Detailed description

  • All the time ranges used in this interface (whether it is a time range defined by a request’s from and to parameters or the subintervals in the return results) follow the “Left closed and right open” principle. That is, these time ranges include the start time, but not the end time. If the from and to values are the same, the time range is invalid and the function will return an error.
  • The subinterval division method for the responses from this interface are consistent and unchanging. If the time range queried by you does not change, neither will the subinterval division in the response.
  • As described above, each call to this interface must return results within a limited time range, and thus each query can only scan a specified number of logs. If a request requires that an extremely large amount of data be processed, the return results for the request may be incomplete (the progress member in the return results indicates whether or not they are complete). At the same time, the server will cache query results within 15 minutes. When a portion of the query request results are cache hits, the server will continue to scan the log data that are not cache hits for this request. To reduce the workload of merging multiple query results, the server cache hit query results and the new hits of the current query are merged and returned to you. Therefore, the log service allows you to call the interface multiple times with the same parameters to finally obtain the complete results. Because the log data volume involved in the query changes massively, the log service API cannot predict how many calls are required to obtain complete results from the interface. Therefore, you must check the progress member value in the return results of each request to determine if they need to continue. You must note that each duplicate call to this interface will consume the same number of query CUs again.

Specific error codes

In addition to general error codes of the log service API, the GetHistograms interface may return the following special error codes:

HTTP Status Code Error Code Error Message Description
404 LogStoreNotExist logstore {Name} not exist. The LogStore does not exist.
400 InvalidTimeRange request time range is invalid. The request time range is invalid.
400 InvalidQueryString query string is invalided The request’s query string is invalid.

The {name} variable in the above error messages will be replaced by the actual logstore name.

Example

Take a project named big-game in Hangzhou as an example. Query the distribution of logs themed groupA in the LogStore named app_log. The query ranges from 2014-09-01 00:00:00 to 2014-09-01 22:00:00, and the keyword is “error”.

Request example:
  1. GET /logstores/app_log?type=histogram&topic=groupA&from=1409529600&to=1409608800&query=error HTTP/1.1
  2. Authorization: <AuthorizationString>
  3. Date: Wed, 3 Sept. 2014 08:33:46 GMT
  4. Host: big-game.cn-hangzhou.log.aliyuncs.com
  5. x-log-bodyrawsize: 0
  6. x-log-apiversion: 0.6.0
  7. x-log-signaturemethod: hmac-sha1
Response example:
  1. HTTP/1.1 200 OK
  2. Content-Type: application/json
  3. Content-MD5: E6AD9C21204868C2DE84EE3808AAA8C8
  4. Content-Type: application/json
  5. Date: Wed, 3 Sept. 2014 08:33:47 GMT
  6. Content-Length: 232
  7. x-log-requestid: efag01234-12341-15432f
  8. {
  9. "progress": "Incomplete",
  10. "count": 3,
  11. "histograms": [
  12. {
  13. "from": 1409529600,
  14. "to": 1409569200,
  15. "count": 2,
  16. "progress": "Complete"
  17. },
  18. {
  19. "from": 1409569200,
  20. "to": 1409608800,
  21. "count": 1,
  22. "progress": "Incomplete"
  23. }
  24. ]
  25. }

In this response example, the server divides the entire Histogram into two equal time ranges: [2014-09-01 00:00:00, 2014-09-01 11:00:00) and [2014-09-01 11:00:00, 2014-09-01 22:00:00). Because the amount of your data is too large, the first query will return incomplete data. The response results indicate that the number of log hits is 3, but the overall results are incomplete. Only the results for the time range [2014-09-01 00:00:00 2014-09-01 11:00:00) are complete with 2 hits, while the results for the other time range are incomplete with 1 hit. In this situation, if you wish to obtain the complete results, you must duplicate the above call request several times until the “progress” member for the overall results changes to “Complete” (as shown in the response below):

  1. HTTP/1.1 200 OK
  2. Content-Type: application/json
  3. Content-MD5: E6AD9C21204868C2DE84EE3808AAA8C8
  4. Content-Type: application/json
  5. Date: Wed, 3 Sept. 2014 08:33:48 GMT
  6. Content-Length: 232
  7. x-log-requestid: afag01322-1e241-25432e
  8. {
  9. "progress": "Incomplete",
  10. "count": 4,
  11. "histograms": [
  12. {
  13. "from": 1409529600,
  14. "to": 1409569200,
  15. "count": 2,
  16. "progress": "Complete"
  17. },
  18. {
  19. "from": 1409569200,
  20. "to": 1409608800,
  21. "count": 2,
  22. "progress": "complete"
  23. }
  24. ]
  25. }
Thank you! We've received your feedback.