All Products
Search
Document Center

Design a timeline structure to improve query efficiency

Last Updated: Sep 04, 2020

Timeline

In TSDB, a timeline consists of one metric and a set of tags. In each timeline, time series data is collected based on continuous time points. One of the timeline examples is {“metric”: “cpu”,”tags”: {“site”: “et2”, “ip”: “1.1.1.1”, “app”: “TSDB”}.In this example, the timeline consists of the cpu metric and three tags. Time series data is generated based on this timeline, and the time series data is displayed as the (timestamp,value) key-value pairs. The following figure shows three timelines.timelines

Inverted indexes for timelines

TSDB generates inverted indexes for each timeline to improve query efficiency. Specifically, TSDB generates an index for each tag key-value pair and metric in a timeline. You can use the indexes to find the timeline that hosts the tag key-value pair and the metric. TSDB generates the following inverted indexes for the preceding three timelines.Inverted indexes

Best practices

  • Reduce the number of timelines

    If the following elements are the same for multiple timelines, the timelines are considered as the same timeline in TSDB:

    • Metric
    • The number of tags
    • The tag key and the tag value of each tag
    • Field name (applicable only when a multi-value data model is used)

      If you use the single-value data model, the upper limit for the number of timelines is the Cartesian product of a metric set and a tag set. If you use the multi-value data model, the upper limit for the number of timelines is the Cartesian product of a metric set, a tag set, and a field set. Therefore, we recommend that you reduce the number of metrics or tags when you design the timelines. This helps you ensure that the number of timelines does not exceed the upper limit.

      When you design the timelines, you may pay less attention to the number of tag values. To reduce the number of timelines, we recommend that you reduce the number of tag values if possible. For example, when you design timelines for a monitored process, we recommend that you do not specify process IDs as tag values. This is because the process ID may change after the process is restarted. You must also use caution if you specify timestamps as tag values. If the number of tag values is large, the number of timelines may show a significant increase. This occurs even if only a limited number of metrics and a limited number of tag keys are used.

  • Reduce the number of timelines indexed by a single tag

    We recommend that you do not specify the same tag for multiple timelines. Otherwise, an excessively large number of timelines are indexed by the tag. This adversely affects query performance.

  • Reduce the number of timelines scanned in a query

    If the set of timelines indexed by Tag 1 is included in the set of timelines indexed Tag 2, you can use only Tag 1 in a query to improve query efficiency. For example, you can query data in timeline 1 described in the preceding example by using the following methods:

{“metric”: “cpu”,”tags”: {“site”: “et2”, “ip”: “1.1.1.1”, “app”: “TSDB”} or{“metric”: “cpu”,”tags”: {“ip”: “1.1.1.1”}.For the second method, only the ip=1.1.1.1 tag is specified, and only one timeline is indexed by the tag. Therefore, this method offers better query performance.