All Products
Search
Document Center

Query based on data preprocessing

Last Updated: May 24, 2020

To ensure compatibility with previous APIs, SDKs, and Grafana, TSDB introduces a tag (granularity) as a query hint to implement queries based on time series preprocessing. If the conditions specified by the hint are met, the system returns the results based on time series preprocessing. Otherwise, the system returns results based on raw data.

To query data based on time series preprocessing, you must specify the tagk=”granularity tag, and the method of preprocessing tag values.

1. You can specify the rollup interval and the operator for tag value preprocessing in the query request, such as 5m.avg.

The following example shows the body of an HTTP POST request:

  1. {
  2. "start":1560218801118,
  3. "end":1560218867499,
  4. "queries":[
  5. {
  6. "metric":"system.mem.base",
  7. "aggregator":"avg",
  8. "downsample":"5m-avg",
  9. "filters":[
  10. {
  11. "type":"literal_or",
  12. "tagk":"__granularity__",
  13. "filter":"5m.avg"
  14. },
  15. {
  16. "type":"literal_or",
  17. "tagk":"HostType",
  18. "filter":"N38"
  19. },
  20. ]
  21. }]
  22. }

This example queries the timelines where the metric is system.mem.base and the HostType=”N38” tag is included. If you have configured preprocessing rules for the timeline, the granularity=”5m.avg” tag takes effect. This allows the database to read the preprocessed data instead of the raw data.

2. You can set the granularity parameter to auto for tag value preprocessing. The system automatically sets the rollup interval and the operator based on the rollup rule.

The following example shows the body of an HTTP POST request:

  1. {
  2. "start":1560742264,
  3. "end":1560742564,
  4. "queries":[
  5. {
  6. "aggregator":"avg",
  7. "downsample":"1m-sum",
  8. "metric":"test.rollup.t5",
  9. "tags":{
  10. "tagk1":"tagv1",
  11. "tagk2":"tagv2",
  12. "tagk3":"tagv3",
  13. "__granularity__":"auto"
  14. }
  15. }
  16. ]
  17. }

This example queries the timelines where the metric is system.mem.base and the tagk1=”tagv1”, tagk2=”tagv2”, and tagk3=”tagv3” tags are included. By default, the time interval is 1 minute and the aggregator is sum for time series preprocessing rules. If you have configured preprocessing rules for the timelines and the rules use the default time interval and aggregator, the system queries the preprocessed data based on the “1m.sum” downsampling rule. This downsampling rule is equivalent to the granularity=”1m.sum” tag.

3. Check whether rollups are used based on the query result

To check whether a database has read the preprocessed data, you can check whether the JSON-formatted response contains rollupGranularity. In the following example, the response contains “rollupGranularity”=”5m.avg”, which indicates that the database reads the preprocessed data.

  1. [{
  2. "metric":"system.mem.base",
  3. "tags":{"HostType":"N38", ...},
  4. "rollupGranularity":"5m.avg",
  5. "dps":{
  6. "1560908100":26.677781312753666,
  7. "1560908160":29.156335115229304,
  8. ...
  9. }
  10. }]

Note that if you do not configure preprocessing rules for a timeline, the system ignores the granularity tag, and returns raw data. Therefore, if you specify an invalid metric or tag in a query, the system cannot find the preprocessing rule. As a result, the raw data points are returned.